1
0
Fork 0
mirror of https://github.com/retspen/webvirtcloud synced 2026-11-11 12:46:49 +00:00

perf(storages): an ISO uploads to an SSH compute without waiting for each write

Each 32 MiB chunk went out as 1024 SFTP writes, each waiting for its
answer: 19 MiB/s on a gigabit LAN, ~1.9 s per chunk with the SSH session.
The writes now go out without waiting and their answers are read as they
come and at the end: ~0.53 s per chunk, the same file (checked by sha256).

paramiko's own pipelined writes (set_pipelined) drop the errors in their
answers, so a full disk would leave a short ISO behind a successful
upload. Each write is registered to its file instead, the way paramiko
does for prefetched reads: every answer, in any order, reaches the file
and an error is raised as soon as one is read.
This commit is contained in:
catborise 2026-10-09 13:15:55 +03:00
parent 24231813d7
commit a87fbf739f
4 changed files with 136 additions and 5 deletions

View file

@ -523,6 +523,25 @@ class StorageUploadSecurityTests(TestCase):
def read(self):
return bytes(self.buf)
# the SFTP client calls of a pipelined write: the file is its own client
MAX_REQUEST_SIZE = 32768
handle = b"handle"
sock = MagicMock(recv_ready=lambda: False)
@property
def sftp(self):
return self
def _async_request(self, fileobj, kind, handle, offset, data):
self.seek(offset)
self.write(data)
def _finish_responses(self, fileobj):
pass
def _check_exception(self):
pass
def seek(self, pos):
self.pos = pos