Experimentation Works: The Surprising Power of Business Experiments
Harvard Business School studied the extent to which organizational values and rewards and an individual’s status affected experimentation.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
transaction data, but that information provides clues only about past behavior, not about how customers might react to future changes.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Having an online testing capability is an important competitive advantage, provided managers know how to use it.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
if management decides to go with third-party testing tools, the decentralized model may just help them get things off the ground quickly, learn effectively, and scale the number of tests.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
determining the best short-term metrics that are good predictors of long-term outcomes is difficult.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
the pursuit of knowledge about cause and effect;
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
One vice president of e-commerce observed that convincing senior leaders of the importance of online testing was the biggest obstacle to scaling experiments.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Consequently, somewhere around 50 percent of tests were winners—an unusually high success rate that should have set off alarm bells. Did the specialists run true experiments that result in gains and losses? Or did they verify something they already knew would work?
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
If you thought something was going to happen and it didn’t, then you’ve learned something meaningful.