Where the milliseconds actually go

Latency is not one number but a sum of four different delays, and they behave completely differently. Propagation delay is the time the signal needs to cover the distance, fixed by the speed of light in the medium. Transmission delay is the time to push all the bits of one packet onto the link, fixed by the link rate and the packet size. Queuing delay is the time a packet waits in a buffer behind other packets, fixed by how busy the network is right now. Processing delay is the tiny time a router spends looking at a header and choosing an exit. Only two of the four respond to money in an obvious way, and the biggest one does not respond at all.

London to New York        5 550 km    one way
speed in fibre           200 000 km/s
one way delay            27.8 ms
minimum round trip       55.5 msThe same calculation for any pair of cities: distance, distance, distance.

Light in vacuum travels 300 000 km per second. In optical fibre the core has a refractive index around 1.47, so the signal moves at roughly 200 000 km per second, about two thirds of that. This is not a limitation of the equipment. It is a property of the glass, and no amount of engineering removes it. Every kilometre therefore costs about 0.005 ms one way, and because a conversation needs an answer, it costs about 0.01 ms per kilometre round trip. That number is the floor under every network design in the world.

Transmission delay depends on how many bits you are sending and how fast the link is, and on nothing else. A maximum size Ethernet frame is 1500 bytes, which is 12 000 bits. On a 1 Gbit/s link it takes 0.012 ms to place on the wire, on a 100 Mbit/s link exactly ten times longer. Notice what this means: for a single small packet the link rate is almost irrelevant next to a 50 ms ocean crossing. Bandwidth decides how long a large transfer takes; it does not decide how long one request takes.

1500 bytes = 12 000 bits

1 Gbit/s    serialisation  0.012 ms
100 Mbit/s  serialisation  0.120 msTwelve thousand bits, two very different links.

Routers do not cut through packets. A router has to receive the whole packet, check it, decide where it goes and only then start transmitting it on the next link. So the total time is a sum over hops, not a maximum, and every hop adds both its own transmission delay and its own processing. On a path of twenty hops that is twenty small additions, which is why a fast local network still loses tens of milliseconds over a long route.

Queuing is the only one of the four delays that is genuinely unpredictable, and it is the reason two identical requests can take very different times. When a link is almost idle, a packet waits for nothing. When the link is loaded, packets line up in a buffer, and that wait can be microseconds or hundreds of milliseconds depending on traffic you do not control. When the buffer overflows, packets are dropped, which for TCP means retransmission, which means at least one extra round trip. This is also why a link can look fast during a test at 3 a.m. and slow during a video call at 8 p.m.

What users feel is round trips, not one-way delay, because almost every protocol is a conversation. A DNS lookup is one round trip, opening a TCP connection is one, a TLS handshake on a fresh connection is one or two more, and only then does the request for the page travel. Opening a page with a cold connection therefore pays three or four round trips before the first useful byte arrives, and each of them costs the full distance to the server and back.

$ curl -w "dns %{time_namelookup} connect %{time_connect} tls %{time_appconnect} total %{time_total}\n" -o /dev/null -s https://example.org
dns 0.004 connect 0.058 tls 0.117 total 0.171

$ curl -o /dev/null -s -w "total %{time_total}\n" https://example.org
$ curl -o /dev/null -s -w "total %{time_total}\n" https://example.org
total 0.171
total 0.062Measuring the phases of a real request, then measuring it again on a reused connection.

This is why connection reuse is not an optimisation but the main event. Keeping a connection open lets the client skip the TCP and TLS handshakes entirely for the next request, turning a 170 ms page load into a 60 ms one across an ocean. It is also why HTTPS costs more than HTTP on a cold connection and almost nothing on a warm one, and why a server that closes idle connections too eagerly makes everything feel slower.

The one real answer to propagation delay is to shorten the distance. A CDN keeps copies of content in many locations and answers each user from the nearest one, so a request that used to cross an ocean now crosses a city. It cannot help with dynamic data that must be produced in one specific place, and it cannot make the first connection to a cold cache fast. What it does is turn a physics problem into a logistics problem, and logistics can be solved.