> Cloud sells egress bandwidth at an enormous markup. Meanwhile ingress is free. If this were a technical limitation ingress would be billed too because there's nothing asymmetric about the Internet.
Symmetry is the entire problem.
1) to maintain settlement-free peering, you need traffic-symmetry. Amazon — on their own - transmits far more data than they receive. They benefit far more on this account from ingress traffic than more egress. Hence, the economic signal that ingress is free.
2) If you have unbalanced network traffic, your scaling factor becomes the transmit direction that is more heavily utilized. A 100gbps circuit can generally send and receive 100gbps simultaneously. But it can’t be used to send 150gbps and receive 50gbps. So if you have a bunch of people all wanting to transmit from your circuit far more than they receive, you have to upgrade your circuits and routers earlier than you otherwise would for a given aggregate amount of traffic. This is expensive, but adding more ingress traffic is free.
Assuming that's all true, it still doesn't explain the magnitude of the markup. VPS providers can do $0.01/gig outbound almost across the board. This includes some very large ones, and because bandwidth is cheap there people tend to host a lot of high bandwidth things there.
GCP, AWS, etc. are roughly 10X more expensive than this. Are they 10X better?
The cost is very high. I can't stress this enough. A terabyte outbound at AWS is roughly $80. For around $300/month I can get 1gbps unmetered at a highly regarded high quality bare metal provider and transmit well over 200TiB in a month. I know because I've done it continuously for years. It works fine. That means AWS is marking up bandwidth by at least 50X at a minimum.
(The above are rough numbers, but they capture the magnitude of the difference.)
If what you say is true and the markup is reasonable then all these companies should be out of business. Either that or they're overselling bandwidth by a giant margin, but that doesn't check out because they specifically advertise as places to put your extremely high bandwidth work loads. If your business model depends on people not doing X, don't advertise as a place for people to do X.
Note: at no point in my comment did I say I thought the specific dollar amount of the bandwidth markup was reasonable or unreasonable. I pointed out reasons why differential bandwidth pricing for ingress/egress exists at the major cloud providers, when you claimed there were no technical reasons.
Also note: VPS providers generally do not exist at the scale where they could or would pursue settlement-free peering. They buy transit from one (or, hopefully, more) upstream providers who have to worry about the first point I made. The VPS providers still have to worry about point #2, but they can do that by courting different kinds of customers, without the baggage (in the context of running a network) of a big legacy content distribution business that both Amazon and Google have.
Symmetry is the entire problem.
1) to maintain settlement-free peering, you need traffic-symmetry. Amazon — on their own - transmits far more data than they receive. They benefit far more on this account from ingress traffic than more egress. Hence, the economic signal that ingress is free.
2) If you have unbalanced network traffic, your scaling factor becomes the transmit direction that is more heavily utilized. A 100gbps circuit can generally send and receive 100gbps simultaneously. But it can’t be used to send 150gbps and receive 50gbps. So if you have a bunch of people all wanting to transmit from your circuit far more than they receive, you have to upgrade your circuits and routers earlier than you otherwise would for a given aggregate amount of traffic. This is expensive, but adding more ingress traffic is free.