Why queues blow up

Think of one till in a shop. People arrive now and then. The till serves one at a time. If people arrive slower than the till can serve them, no queue builds up. If they arrive faster, the queue grows for ever. The interesting case is in between.

Say the till can serve ten people an hour and nine arrive. The till is busy 90% of the time. You might expect a short queue. In fact the wait is long, because arrivals are not spread out. Some hours five people come at once and the till has no spare time to catch up.

Push it to 95% and the wait roughly doubles. Push it to 99% and it is ten times the wait at 90%. Nothing broke. No one did anything wrong. The curve is simply shaped like that: as you get close to full use, the wait rises very fast.

There is a second rule that is easier. The number of people in the shop equals how fast they arrive times how long each one stays. If six people join the queue each hour and each is in the shop half an hour, there are three people in there at any moment. You can rearrange that to work out any one of the three from the other two, and it holds whatever the queue is made of.

This is why a road at 95% of its capacity jams when one driver brakes, and why a server at 95% of its processor falls over at the first spike. It is also why the fix is not always a faster till. Smoothing the arrivals, or shortening the time each person takes, can do more than adding capacity.

The practical lesson is to leave slack on purpose. A system run at 70% looks wasteful on a spreadsheet. It is the slack that keeps the wait short when the day is not average, and no day is average.