For the last several years, I’ve been at the forefront of a movement known as implementation science, or the science of scaling. In this work, we are trying to understand why some products, companies, and social programs thrive as they grow, while others peter out.
When a seemingly promising idea loses efficacy or profitability as it expands, we call it a “voltage drop.” These failures to scale never happen because of one single reason.
Over the last 25 years as a behavioral economist, consultant to companies large and small, and former White House economic adviser, I’ve identified five causes, what I call “vital signs,” of voltage drops.
1. False Positives
This occurs when you interpret a piece of evidence or data as proof that something is true, when in fact it isn’t — for example, as we’ve seen with inaccurate Covid test results. For scaling, a false positive is an erroneous sign that an idea has voltage when it really doesn’t.
Sometimes a false positive occurs because of a statistical error, as was the case with the famous drug abuse prevention program, DARE After an independent study showed promising short-term results, the program received an influence of funding from the US Department of Justice. However, there were several problems with the study: It excluded drugs like alcohol and marijuana, focusing on tobacco; it was based on a small sample size; and later studies and even meta-analyses could not replicate the results.
In other cases, false positives result from intentional lying. Think of Elizabeth Holmes and the purportedly groundbreaking blood-testing technology of Theranos, which didn’t actually exist.
When possible, the solution for rooting out false positives is to have at least three independent replications of the idea that show early promise. In companies with confidential research, employees must be incentivized with financial rewards that encourage them to question results.
2. Biased Representativeness of Population
Once you’ve reliably demonstrated the efficacy of the endeavor you hope to scale, the next step is to answer the question “How broadly will the idea work?”
All ventures must understand their potential audience. The first way to do this is by making sure your test samples in the small scale reflect the larger population at scale. Otherwise, you’ll be like McDonald’s, which fell victim to selection bias when it launched the unsuccessful Arch Deluxe. Focus group participants loved the new product, but they weren’t representative of the majority of Americans, who simply wanted to keep eating their Big Macs.
To weed out such biases, make sure your early adopters are a random sample. You should also make sure that your survey respondents have appropriate incentives to tell you the truth. A focus group participant who says they would purchase a product if it was introduced could simply be saying, “I would love the option to consider that product in the future,” as opposed to “I will be purchasing the product in the future.”
3. Non-Negotiables That Can’t Grow or Be Replicated
For an idea or enterprise to hold strong at scale, you need to know whether your “non-negotiables” — the drivers of your success — can be replicated at scale. In other words, is your secret sauce the “chef” or the “ingredients”? Since people don’t scale well (ie, they can’t be cloned), talent-centric ventures often don’t either. You can’t afford all the talent you need as you grow, so you hire fewer high-performers and quality suffers at scale — a cruel voltage drop.
But, this vital sign is about much more than just people. As you scale, regulatory constraints, resource constraints, fidelity concerns, and a host of other issues might arise. In the end, we must bring these scaling constraints back to the petri dish and make sure the idea works with them in place.
4. Negative Spillovers
A spillover effect is the unintended impact one event or outcome can have on another event or outcome. A classic example is when a city opens a new factory, and the air pollution it produces impacts the health of nearby residents.
As you scale, the likelihood of spillovers increases dramatically. General equilibrium effects, or natural readjustments of the market, are one chief cause. I saw this first hand when I was the chief economist at Uber. A coupon that led to more riders in one area of Seattle failed when we scaled it to the whole city because prices kicked in, and users found cheaper ways to get around that night.
Positive spillovers exist too, like network effects that make a social-media platform more valuable as more people join it. When designing your idea early on, you must anticipate negative spillovers and look for opportunities to engineer and benefit from positive ones.
5. The Cost Trap
To scale successfully, you need to determine not only how many people like your idea, but also what they’re willing to pay for it and, crucially, how much it will cost to provide.
When designing your enterprise, you must account for two types of costs: upfront fixed costs, like the one-time investment for the research and development to create a new product or service, and your ongoing operating expenses. Upfront costs can be recovered, but operating ones can bleed you and lead to a voltage drop, as happened to the innovative scientific wellness startup Arivale, which was poised to change preventative health care, only to go bankrupt a few years laterbecause it couldn’t find a viable price point for its services.
One strategy to escape the cost trap of scaling is to make sure you benefit from economies of scale, a skill Elon Musk excels at in all his ventures. Ever since he has helped transform the world of online banking at PayPal, each major innovation he has undertaken thrives on scale economies. Consider Tesla. Its massive success can be traced to economies of scale of its two most important components: batteries and solar power generation cells, both of which can be manufactured significantly cheaper in higher numbers. In addition, everything at Tesla is geared toward increasing the efficiency of “the machine that makes the machines,” or what Musk affectionately calls his “Alien Dreadnought” — that is, a highly advanced, fully automated production facility.
Another strategy is to create models that don’t rely on top-tier talent. As you scale, finding and paying high-performers will become prohibitive. The solution is to create products that can give their full value to customers even with average performers delivering it.
When I give talks on this topic, I like to invoke the famous opening line of Leo Tolstoy’s novel Anna Karenina: “Happy families are all alike; every unhappy family is unhappy in its own way.” Similarly, scalable ideas are all alike; every unscalable idea is unscalable in its own way. The difference with scaling is there are only five main obstacles you face. And once you anticipate and avoid them, you can scale your idea for the highest voltage possible.