Averages that lie

Take the height of a hundred people. The average tells you a lot. Nobody is twice the average height. Add the tallest person alive and the average barely moves. Heights cluster, and the cluster has a middle.

Now take the wealth of a hundred people, and let one of them be a billionaire. The average wealth is now in the millions, and ninety-nine of them are nowhere near it. The average is a true number that describes nobody in the room.

The difference is not fussiness. It is two different shapes of world. In the first, cases pile up around a middle and the extremes are mild. In the second, a few cases are so large that they carry most of the total. Book sales, city sizes, wars, website visits and software outages mostly live in the second world.

Three habits follow. Use the middle value, the one with half above and half below, rather than the average. Ask what the biggest case was, because in the second world it may be most of the story. And be slow to trust an average built from a short run of data, because the big cases are rare and you may simply not have seen one yet.

This is also why sample size can fool you. In the second world a small sample usually misses the extremes, so it looks calmer than the truth. The average of your sample creeps upward as you collect more, which is a strange thing to watch and a sign of what you are dealing with.

The test is simple. Ask whether one case could be bigger than everything else put together. If it could, be careful with averages. If it could not, the average is probably fine and you can stop worrying about it.