The Seven Pillars of Statistical Wisdom

The second pillar, Information Measurement, is logically related to the first: If we gain information by combining observations, how is the gain related to the number of observations?

A Preliminary Discourse on the Study of Natural Philosophy. Herschel gave particular emphasis to what he called residual phenomena.

Galton was, in essence, able to establish some of the practical consequences of Mendelian genetics without the benefit of knowing any genetics, nearly two decades before Mendel’s work was rediscovered.

In order to make the case for evolution by natural selection, it was essential to establish that there was sufficient within-species heritable variability:

Granted that more evidence is better than less, but how much better? For a very long time, there was no clear answer.

combining objects of the same generic group that do not cluster towards a common centre; no more should we attempt to compose generic portraits out of heterogeneous elements, for if we do so the result is monstrous and meaningless.”

Clearly the calculation of a single probability is not the answer to all questions. A probability itself is a measure and needs a basis for comparison. And clearly some restriction on allowable hypotheses is needed, or else a self-fulfilling hypothesis such as “the data are preordained” would give probability one to any data set.

given a number of observations, you can actually gain information by throwing information away!

with exactness, and subducted, the residual facts are constantly appearing in the form of phenomena altogether new, and leading to the most important conclusions.