The Data Detective

They call themselves the Cochrane Collaboration and they maintain the Cochrane Library, an online database of systematic research reviews.

we should compare the fallibility of today’s algorithms with that of the humans who would otherwise be making the decisions.

Curiosity breaks the relentless pattern.

Before I repeat any statistical claim, I first try to take note of how it makes me feel. It’s not a foolproof method against tricking myself, but it’s a habit that does little harm and is sometimes a great deal of help. Our emotions are powerful. We can’t make them vanish, nor should we want to. But we can, and should, try to notice when they are c

Third, superforecasters tended to update their forecasts frequently as new information emerged, which suggests that a receptiveness to new evidence was important. This willingness to adjust predictions is correlated with making better predictions in the first place:

the tactic of simply refusing to collect basic statistics could only make sense for a libertarian, laissez-faire regime. And the truth is that very few people seem attracted by that prospect. For better or worse, we want our governments to take action, and if they are to take action they need information. Statistics collected by the state make for

a statistical metric may be a pretty decent proxy for something that really matters, but it is almost always a proxy rather than the real thing. Once you start using that proxy as a target to be improved, or a metric to control others at a distance, it will be distorted, faked, or undermined. The value of the measure will evaporate.

The importance of the base rate was made famous by the psychologist Daniel Kahneman, who coined the phrase “the outside view and the inside view.” The inside view means looking at the specific case in front of you:

just as the most brilliant thinkers of the age failed to make progress while practicing in secret, secret algorithms based on secret data are likely to lead to missed opportunities for improvement.