Gary Loveman, the former CEO of Caesars Entertainment and an early advocate for running experiments in the hotel and casino business, recognized that adding controls doesn’t come naturally to people. In an interview with MIT Technology Review, he explained: Let’s say that one of our properties had lower revenues than they’d like, and they think

the analysis of big data using standard mathematical methods, such as regression analysis, results mostly in insights about correlation but not causation.

In an ideal experiment, the tester separates an independent variable (the presumed cause) from a dependent variable (the observed effect) while holding all other potential causes constant,

Having an online testing capability is an important competitive advantage, provided managers know how to use it.

some companies are limited to running bigger experiments because they don’t have enough customer traffic to test very small changes. But even here, an “experiment with everything” approach can help us explore new problem and solution spaces.

the jeopardy thesis is perhaps the most dangerous, because it asserts that a proposed action, though beneficial, involves unacceptable risk and costs.

Harvard Business School studied the extent to which organizational values and rewards and an individual’s status affected experimentation.

Hirschman concluded that arguments directed against progress usually come in three flavors: the perversity thesis, the futility thesis, and the jeopardy thesis.

changing the organization was necessary to increase the scale and impact of online experiments.