Experimentation Works: The Surprising Power of Business Experiments
experiments can be run in live environments, on real customers, with (nearly) 100 percent fidelity.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
By combining the power of software and the rigor of controlled experiments, companies can turn themselves into learning organizations—turbocharged! But to unleash this power, you need to build an experimentation organization that masters the science of testing and has a culture, processes, and management system that contradicts what we value today.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
It is impossible to have an experimentation culture if “not winning” equates to “losing.” At the same time, an experimentation culture is not a “break everything” organization either, wherein anything and everything can be thrown against the wall in the name of creative disruption.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Google’s experts missed their mark 96.1 percent of the time. But it’s precisely that capability—to test what does and does not work at a huge scale—that has given the company an advantage against its competitors.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
transaction data, but that information provides clues only about past behavior, not about how customers might react to future changes.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Clay Christensen found that successful managers “planned to fail early and inexpensively in the search for the market for a disruptive technology. They found that their markets generally coalesced through an iterative process of trial, learning, and trial again.”
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
“The key to the Post-it adhesive was doing the experiment.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
running experiments should be as normal as running the numbers.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
redesign of its Snapchat multimedia messaging app, cofounder and CEO Evan Spiegel, an executive with extensive design training, was reluctant to subject the new user experience to rigorous experiments and slow down its release. And when user sentiment dropped 73 percent, he insisted: “Even the complaints we’re seeing reinforce the [design] philosop
... See moreStefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
.case-study
Consequently, somewhere around 50 percent of tests were winners—an unusually high success rate that should have set off alarm bells. Did the specialists run true experiments that result in gains and losses? Or did they verify something they already knew would work?