Experimentation Works: The Surprising Power of Business Experiments
learning laboratories where decisions are grounded in facts.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
hundreds of other changes per month that improve the user experience, are the major reason that Bing is profitable and its share of US searches conducted on personal computers has risen to nearly 23 percent
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Noise occurs when variables are not or cannot be controlled, or when too many variables are being manipulated at once because an experiment is too expensive and many variants are crammed into one or a few tests. In either case, it’s difficult to discern cause and effect.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
By combining the power of software and the rigor of controlled experiments, companies can turn themselves into learning organizations—turbocharged! But to unleash this power, you need to build an experimentation organization that masters the science of testing and has a culture, processes, and management system that contradicts what we value today.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
series of A/B tests in which artificial delays were added to study the effects of tiny differences in loading speed. The data showed that every 100-millisecond difference in performance had a 0.6 percent impact on revenue.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Experimentation helps us begin to answer the kinds of questions that all organizations confront: How do we know what products to make, what customer experiences to offer, and what
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
experiments can be run in live environments, on real customers, with (nearly) 100 percent fidelity.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
“Show me the experiment!” instead of acting only on conventional wisdom, focus group findings, and even data analysis.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
The attractiveness of using computer simulation can be found in the higher speed and efficiency of carrying out iterations. Teams can win, fail, and learn fast.