Series: MIT Press
Hardcover: 256 pages
Publisher: The MIT Press; 1 edition (September 12, 2014)
Language: English
ISBN-10: 0262028360
ISBN-13: 978-0262028363
Product Dimensions: 6 x 0.5 x 9 inches
Shipping Weight: 1.1 pounds (View shipping rates and policies)
Average Customer Review: 3.9 out of 5 stars See all reviews (84 customer reviews)
Best Sellers Rank: #97,517 in Books (See Top 100 in Books) #19 in Books > Business & Money > Human Resources > Knowledge Capital #133 in Books > Health, Fitness & Dieting > Psychology & Counseling > Creativity & Genius #164 in Books > Business & Money > Processes & Infrastructure > Organizational Learning
My team provides the experimentation platform for most online experiments at Microsoft (e.g., search for exp-platform), and I have been running online experiments for 12 years now, with Bing running about 10,000 experiments per year, so I know something about online controlled experiments. Michael Schrage quotes me a few times in the book, and encouraged me to write an unbiased review.The book is written extremely well, and the overall message is absolutely spot on: most organizations would benefit from running cheap experiments and iterating quickly. In many cases, organizations get into analysis-paralysis instead of running simple experiments; worse, deep analyses often come late and have low predictive accuracy.The 5x5 approach advocated by Michael Schrage says "Give a diverse team of 5 people no more than 5 days to come up with a portfolio of 5 business experiments that cost no more than $5,000 (each) and take no longer than 5 weeks to run."My key observation is that the author proposes a solution with a plan to a problem, and violates his own advice to avoid such solutions. He's asking the reader to believe in exactly what he's asking others to suspend doing. There are many possible ways to get an organization to be data-driven. Is 5x5 better than others? Having been humbled by many good ideas that don't perform as well as people hope, I'm not sure there's data to say that the 5x5 proposal is dominating.In the paper "Online Controlled Experiments at Large Scale" there's a section on the "Cost vs. Benefit and the Ideas Funnel" where prioritization of ideas is discussed, a mechanism that aligns more with the author's statement that "Experiments need to be seen and managed as investments" and less with the 5x5 approach.
I am sure the author is a very effective consultant. He knows his stuff. He has a lot of experience. But, this narrow subject just does not warrant a whole book. A 10 page essay would have more than sufficed. The writing is so repetitive because the author struggles to spread 10 pages worth of content over 200 pages of print. It just does not work. The title of my review indicates the author repeated his 5x5 concept a hundred times throughout the book. And, that's probably a lowball estimate. But, the book has other weaknesses.There is nothing magic about the number 5. Although 5x5xetc... has a good ring to it, there is an unlimited way of gathering resources to conduct short-term, rapid feedback experiments. You can indeed try 5 different concepts with teams of 5 people each costing no more than $5,000, over a 5-week period. But, many innovations have been done with a single person, trying a single concept, costing close to nothing, with near immediate feedback in this Internet age. The single innovator may succeed not by trying very different concepts but by iteratively honing and tweaking his original concept and watch live the performance of his last iteration. That's a similar approach the author indicates that Edison, Jobs, , and other innovators have used. So, the author clearly understands this. But, it does contradict his 5x5 dogma.Regarding designing rapid low cost experiments, there may be some avenues the author did not explore. Within the book, I did not see any mention of internal prediction markets that are used by many hi-tech companies to quickly differentiate what projects are successful.
As I began to read Michael Schrage's latest book, I was again reminded of a passage in Paul Schoemaker's latest book, Brilliant Mistakes: "The key question companies need to address is not `[begin italics] Should [end italics] we make mistakes?' but rather [begin italics] Which [end italics] mistakes should we make in order to test our deeply held assumptions?'" This is precisely what Anjali Sastry and Kara Penn have in mind when introducing, in Fail Better, what they characterize as a better approach to innovation: designing smart mistakes, learn from them, and thereby achieve greater success and do so sooner.Peter Sims has much of value to say about this strategy in Little Bets: How Breakthrough Ideas Emerge from Small Discoveries. As he explains, "At the core of this experimental approach, little bets are concrete actions taken to discover, test, and develop ideas that are achievable and affordable. They begin as creative possibilities that get iterated and refined over time, and they are particularly valuable when trying to navigate amid uncertainty, create something new, or attend to open-ended problems."In Serious Play (1999), Schrage introduces several core concepts that he develops in much greater depth in this, his latest book. The 5x5 X(experimental)-team approach is a rapid innovation methodology emphasizing lightweight, high-impact experimentation, as follows: "Give a diverse team of 5 people no more than 5 days to come up with a portfolio of 5 business experiments that cost no more than $5,000 (each) and take no longer than 5 weeks to run." Schrage adds, "the 5x5 seeks an 80/20/20 vision.
The Innovator's Hypothesis: How Cheap Experiments Are Worth More than Good Ideas (MIT Press) The Innovator's Way: Essential Practices for Successful Innovation (MIT Press) Good Cheap Eats Dinner in 30 Minutes or Less: Fresh, Fast, and Flavorful Home-Cooked Meals, with More Than 200 Recipes Worth Every Penny: Build a Business That Thrills Your Customers and Still Charge What You're Worth How to Do a Year's Worth of Bookkeeping in One Day: Make QuickBooks Do Your Data Entry For You: 2012 - 2018 Versions (How to Do a Year's Worth of Bookkeeping For You) The EXIT Formula: How To Sell Your Business For 3x More Than It's Worth Today The Ultimate Panini Press Cookbook: More Than 200 Perfect-Every-Time Recipes for Making Panini - and Lots of Other Things - on Your Panini Press or Other Countertop Grill Environmental Experiments About Air (Science Experiments for Young People) Creepy Crawlies and the Scientific Method: More Than 100 Hands-On Science Experiments for Children Beyond the Essene Hypothesis: The Parting of the Ways between Qumran and Enochic Judaism The Happiness Hypothesis: Finding Modern Truth in Ancient Wisdom God - the Failed Hypothesis: How Science Shows That God Does Not Exist The Creation Hypothesis: Scientific Evidence for an Intelligent Designer The Lincoln Hypothesis More Than Friends (More Than... Book 1) The $5 a Meal College Vegetarian Cookbook: Good, Cheap Vegetarian Recipes for When You Need to Eat (Everything Books) Cheap Insurance for Your Home, Automobile, Health, & Life: How to Save Thousands While Getting Good Coverage Albert Einstein and Relativity for Kids: His Life and Ideas with 21 Activities and Thought Experiments (For Kids series) Gandhi's Printing Press: Experiments in Slow Reading Canada Dry Alcohol-Free Party Drinks: Recipes & Serving Ideas for More Than 300 Delicious Beverages