Skip to content
Rabin Poudyal
Rabin Poudyal
← All models

Science and physics

Entropy

Left alone, an isolated system drifts into its most probable, least ordered state, so keeping anything ordered costs continual work.

Rudolf Clausius, who stated the second law of thermodynamics in 1850 and coined the word entropy in 1865; Ludwig Boltzmann gave it a statistical meaning.

Clausius stated the second law of thermodynamics in 1850: no process can have as its sole effect the transfer of heat from a colder body to a hotter one. In 1865 he named the quantity behind it entropy, defined for a reversible change as the heat exchanged divided by the temperature. In an isolated system entropy never decreases, and it reaches its maximum at equilibrium. Boltzmann later explained why: entropy measures how many microscopic arrangements produce the same macroscopic state (S = k log W). There are vastly more ways to be mixed, spread out and lukewarm than to be sorted, concentrated and hot in one spot, so a system left to itself wanders into the common states and almost never back.

The transfer is that order is a maintained state, not a resting one. A tidy kitchen, a clean codebase and a readable data warehouse are low-entropy arrangements, and each stays that way only while someone spends effort on it. A repository is not an isolated gas, so the second law does not literally apply to it, but the same statistics do: there are far more ways for a system to be slightly wrong than exactly right, and every unreviewed change, undocumented workaround and unpinned dependency is a random step into the larger space. Engineers called this software rot before most of them had heard of Clausius. The practical rule is to budget for the work of staying ordered: refactoring time, cleanup, pruning.

Two cautions. The metaphor outruns the physics fast. Thermodynamic entropy is a measured quantity in joules per kelvin; the 'entropy' of an organisation is a figure of speech, and the biosphere has built elaborate order for billions of years by taking in sunlight and radiating waste heat to space. Open systems can become more ordered; they need a flow of energy and somewhere to dump the disorder. Second, Shannon's information entropy (1948) borrows the name and the logarithm but measures the uncertainty of a message source, not heat. The two are related through Boltzmann's formula and are not the same thing, so a sentence that slides from one to the other is usually hiding a gap in the argument.

Example

Rudolf Clausius, born on 2 January 1822, announced the second law of thermodynamics in a paper published in 1850 in Poggendorff's Annalen der Physik und Chemie, building on James Joule's finding that heat and work are two forms of what would soon be called energy. In another Annalen paper in 1865 he introduced the word entropy for the quantity whose growth measures how much of a system's energy has become unavailable for doing work. The law has been restated many times since, but every version says the same thing: in an isolated system that quantity never goes down.

Where it applies

Schedule maintenance as a line item instead of hoping it happens: dependency updates, dead-code removal, deduplicating a dataset. In a data pipeline, any schema that is not validated at the boundary drifts, so put the check where the data enters. When something has stayed ordered for years, ask who or what is paying the energy bill, because something is.

Think about it

Which system you rely on is ordered only because someone is quietly maintaining it, and what happens when they stop?

Applied it to something real? Log it.

Test yourselfWhy does an isolated system drift towards disorder rather than order?

Because vastly more microscopic arrangements look disordered than ordered, so random change almost always lands in a disordered state. Keeping order needs energy spent from outside.

Related models

Sources and further reading