SFTP was designed as a remote file system system access protocol rather than transfer a single file like scp.
I suspect that the root of the problem is that SFTP works over a single SSH channel. SSH connections can have multiple channels but usually the server binds a single channel to a single executable so it makes sense to use only a single channel.
Everything flows from that decision - packetisation becomes necessary otherwise you have to wait for all the files to transfer before you can do anything else (eg list a directory) and that is no good for your remote filesystem access.
Perhaps the packets could have been streamed but the way it works is more like an RPC protocol with requests and responses. Each request has a serial number which is copied to the response. This means the client can have many requests in-flight.
There was a proposal for rclone to use scp for the data connections. So we'd use sftp for the day to day file listings, creating directories etc, but do actual file transfers with scp. Scp uses one SSH channel per file so doesn't suffer from the same problems as sftp. I think we abandoned that idea though as many sftp servers aren't configured with scp as well. Also modern versions of OpenSSH (OpenSSH 9.0 released April 2022) use SFTP instead of scp anyway. This was done to fix various vulnerabilities in scp as I understand.
Notably, the SFTP specification was never completed. We're working off of draft specs, and presumably these issues wouldn't have made it into a final version.
Because that is a poor characterization of the problem.
It just has a in-flight message/queue limit like basically every other communication protocol. You can only buffer so many messages and space for responses until you run out of space. The problem there is just that the default amount of buffering is very low and is not adaptive to the available space/bandwidth.
Yeah, it's an issue because there is also the per channel application layer flow control. So when you are using SFTP you have the TCP flow control, the SSH layer flow control, and then the SFTP flow control. The maximum receive buffer ends up being the minimum of all three. HPN-SSH (I'm the dev) normalizes the SSH layer flow control to the TCP receive buffer but we haven't done enough work on SFTP except to bump up the buffer size/outstanding requests. I need to determine if this is effective enough or if I need some dynamism in there as well.
I hate the original Mac Pro case so much!!! The metal box is so heavy and the edges of the aluminum top handle is so sharp that you definitely need a pair of heavy-duty gloves to move it around.
The assumed mentality of “being flexible” is the very reason WireGuard was created to fight against in the first place, otherwise why bother? IPSec is already standardized and with wide-spread hardware implementation (both FPGA and ASIC) and flexible.
What ARM64 machines are you using and what are they used for? Last year you were announcing Gen 12 servers on AMD EPYC (https://blog.cloudflare.com/gen-12-servers/), but IIRC there weren’t any mentions of ARM64. But now it seems you’re running ARM64 in full production.
Yeah but those are pretty dated. I was under the impression those old Ampere servers are not efficient compared to modern EPYC anymore. So I’m wondering what their current generation of arm64 servers look like :p
