Throughput

The number everyone looks at first, and why it answers the smallest question

Throughput is a count: bits moved per unit of time, across a given point, in one direction. A link rated at fifty megabits per second can carry fifty million bits through it every second if everything goes right. When you restrict throughput in a test environment — a shaper inside the kernel, a conditioner profile, a policed egress port — you are governing that count. Nothing more.

That narrowness is why throughput is both the most natural thing to measure and the most routinely overread. Engineers reach for it first because it is concrete and easy to compare. Marketing took the same number for exactly the same reason. Neither group is wrong about what it measures; both groups have at times acted as though it measures more than it does.

What the number actually tells you

A pipe with high throughput can still punish a user. Throughput tells you how many bits moved, not when they arrived. A video conference needs small packets to cross the network within tens of milliseconds of being sent; a fifty-megabit link that delivers them in a hundred-and-twenty milliseconds is useless for that conversation regardless of its rated capacity. The call drops, the grid freezes, the other person's voice arrives in chunks — and the link utilisation graph looks fine the entire time.

This is the elementary distinction between throughput and latency, and it has been in the literature for decades. Van Jacobson's foundational 1988 paper on congestion avoidance and control, written at Lawrence Berkeley National Laboratory in Berkeley, California, treated them as separate problems from the outset: throughput is a property of the path; latency is a property of the queue. His TCP congestion control algorithm — the one that pulled the early internet back from collapse — was designed to protect both, not to trade one against the other.

What sits between a sender and a receiver is never just a pipe. It is a pipe with queues in it. Packets that arrive at a congested interface do not vanish; they wait in a buffer, sometimes a very large one, until the link is ready to forward them. The throughput can remain high because the buffer keeps the link busy, while every packet spends tens or hundreds of milliseconds sitting in that queue before it moves at all. Throughput is unaffected; latency balloons. This mechanism has a name — bufferbloat — and it became a serious concern in commodity equipment as memory prices fell through the 2000s.

Why throttling throughput is not the same as simulating a bad link

When a developer or tester dials down throughput to simulate a slow connection, they are doing something real and useful. They are constraining the rate at which packets leave a queue, forcing the application to operate under scarcity, and revealing whether it handles that scarcity gracefully — whether it buffers video intelligently, whether its retry logic backs off correctly, whether its UI acknowledges the wait.

But the number alone does not reproduce the full character of a poor connection. A dial-up line in the late 1990s was slow in throughput and also high in latency — the modem's training sequence, the acoustic coupling, the compression negotiation all added delay that had nothing to do with the bit rate. A congested mobile link in a busy stadium is slow in throughput but also deeply variable: packets arrive in bursts, then stall, then burst again. Jitter — variation in the gap between packets — can break a real-time audio stream even when average throughput is technically adequate. Loss, which a simple token bucket shaper may not introduce at all, forces TCP to retransmit and can halve effective throughput independent of the nominal rate limit.

The token bucket, the classic mechanism underlying most software rate limiters, works by admitting packets only as tokens accumulate at a set rate, with a small burst allowance. It enforces an average rate faithfully; it says nothing about delay, reordering, or loss. Testing only against a token bucket tells you how the application behaves under scarcity, not how it behaves under adversity. Those are related but different questions.

Sally Floyd, whose work at Lawrence Berkeley National Laboratory extended Jacobson's foundations through the 1990s, was centrally involved in developing random early detection — an active queue management technique that drops packets before a buffer is completely full, giving TCP senders an earlier signal to back off. The important insight is that the drop is the signal. Throughput is a consequence of how senders respond to that signal; it is not itself the lever being pulled. Active queue management disciplines like CoDel, developed by Kathleen Nichols and Van Jacobson and described in a paper published through ACM, work on this same principle: manage the latency in the queue, and throughput takes care of itself.

CoDel, which stands for Controlled Delay, targets a latency threshold rather than a queue length. Its authors' argument was that queue length is the wrong quantity to watch — a short queue on a slow link can mean long delay just as a long queue on a fast link can mean short delay. The IETF published CoDel as RFC 8289, making explicit that the goal was latency control, with throughput as a byproduct rather than a target. That inversion is not a technicality; it reflects a genuine reorientation of what "good" means for a queue.

The right role for a throughput limit

None of this means throughput limits are useless in conditioning or testing. They are often the first thing to set, and correctly so: an application that cannot function at two megabits per second is not ready for a significant fraction of the world's connections, whatever else is true of it. A throughput limit exposes resource exhaustion, premature timeouts, absent progress indicators, and assumptions about bandwidth that the developer never thought to question. Those are real findings.

The mistake is stopping there. Throughput, latency, jitter, and loss each exercise different failure modes in a network stack and in the application above it. A test harness that controls only rate is missing the variables that break real-time communications, adaptive streaming, and anything that depends on TCP's recovery behavior under loss. The full shape of a bad connection — its texture, not just its ceiling — requires all four dials, not one.

Engineers who build network conditioning tools have known this from the start. The applications those tools are meant to test have sometimes needed reminding. Throughput is the easiest quantity to state, to sell, and to measure. It is also, on its own, the one that explains the least about whether a connection actually works.