Why Your Company Won’t Start High-upside projects
Project managers are rarely data science philosophers. They are typically organized, risk-averse professionals whose aspirations consist of keeping a rickety train on the tracks until it reaches a prescribed destination. Typically, this consists of the point in space and time wherein a VP or C-suite exec pats them on the back for admirable completion of a project, confirms that they have delivered upon their responsibilities, and rewards them with bonuses, promotions, and celebratory meals.
Lost in this reductive description is the analysis of the range of possible outcomes. Some projects are straightforward. I call up a guy to have my driveway sealed, agree upon a date and price, then mark my calendar. The range of outcomes includes neither extraordinary upside nor downside. The best outcome is one where I pay ~$100, the guy shows up on time, and a day later, my driveway is prepared to endure another Chicagoland winter. The worst involves some combination of weather delays and a poorly-surfaced driveway wherein I need to pay someone else ~$100 to do the job properly.
Software development projects involve decidedly larger ranges of outcomes, and this is only magnified when the advanced mathematics of data science enters the conversation. Why is this?
Innovation
Among the reasons why startups fetch dizzying valuations and venture capitalists throw money in their direction is their potential to do something new.1 That which is truly original and might truly yield a novel product for humanity offers almost unlimited upside.
The downside, of course, is that attempting to do something new requires experimentation with ideas that may or may not succeed. Each of these experiments contains tasks whose difficulty (or even feasibility) is unknown at their outset.
Intuitively, the more ambitious the aspiration, the larger the required advance from current technology. This means higher upsides and increasing probabilities of failure.
This poses little issue in the aggregate.
The Back Of The Envelope
Consider a hypothetical project with a cost of x. Its probability of “success” is p. The return, if it succeeds, is rx.
The expected value of this project is calculated simply as:
Assuming we wish to undertake projects with a positive expected value (EV > 0), we simply filter for projects where pr > 1, and live wealthily ever after.
But as we began this discussion, we noted that projects have managers responsible for their outcomes. They have their own calculus with which to contend. Let’s say, in the case where a project succeeds, they’ll receive a benefit, β. These bonuses, promotions, and other feathers in professional caps are the carrots which are balanced against the stick—the monetary damage to their career if the project fails to bear fruit. Let’s call this δ.
For these project managers, their expected value looks something like:
If an expected value of 0 represents our break-even point, solving for p yields:
Thus, if p does not exceed δ/(β+δ), the project is a dubious proposition for whichever project manager or VP assumes responsibility for the end result.
Now let’s compare the two break-even factors:
A little rearranging:
Professional Pragmatism
The magnates of the previous century erected factories and residential complexes. The probability of the project succeeding, p, was high. Consider a project with a 90% probability of success, with an r of 1.2. A bonus of, say 25% of a VP’s base salary is offered for success, and if the project goes belly up, it’s 50-50 whether that VP remains employed (δ ~ 0.5) 2
The project kicks off without much debate.
But now, in the world of high-stakes, high-upside software innovation, the calculus is decidedly different. What about a project where the probability of success is 10% (p = 0.1), but the potential return is 100x (r ~ 100)?
The expected value of the project is clearly positive.
But alas:
And though the political machinations and boardroom debates may obfuscate these simple equations, ultimately, companies decrease their own agency by having one person accountable for a project with a 90% chance of failure and a 10% chance of 100x upside. In turn, they eschew opportunities and build a culture of risk-aversion and manipulated data.3
Size Matters
At a small company, this troublesome reality is manageable—a CEO accepts and understands both the risk and the upside and likely has the perspective required to authorize risk-taking and avoid punitive reactions. But the larger the company, the less likely that an individual will be permitted to proceed with an obviously worthwhile endeavor. What if a venture offers a 1% chance of a 1,000,000x upside? Who signs off, accepting accountability for the (probable) loss?
Perhaps, this is why historical returns on venture capital investments are impressive while the number of large corporations willing to place similar wagers internally is so low.
Perhaps, this is why there is value in attempting to hire individuals who typically would never consider a role at any company with more than ten employees.4 Those comfortable with unconventional, high-upside thinking are those who will sprint ahead with low-probability, high-upside aspirations. Should we fire them or empower them? The question is less about probability theory than managerial incentive structures!
Imagine the project manager or VP responsible for contracting with a third-party to develop software with a small probability of generating massive profits. Why risk your career when even in the improbable, best-case scenario, the upside largely fills other pockets?
How can growing companies nurture longtermist, optimistic risk-taking?
Great ideas emerge from junior and senior employees alike. Some instill the passion that leads a CEO to sleep on couches and dine on ramen and tears. At AE, the business serves as co-founder and investor in these nascent ventures, continuing to supply the salary required to procure comfortable lodging and overpriced smoothies.
We invest in these ideas both monetarily and with additional team members as needed to incubate, gestate, and deliver an extraordinary product. We provide guidance, goals, and insight from our collective experience. We provide hosting and marketing resources and offer the founders equity in their products (higher β) If it fails, we don’t fire the employee (lower δ). We accept the calculus above that most companies will not.
Do you want to be a skunkworks founder?
Start by becoming a valuable member of our team—exemplify the core values and excel at your role. Proving yourself reinforces the old startup adage that we invest in the person as much as the idea. We want to invest in you.
Then start hustling. Startup CEOs don’t necessarily need to couchsurf and starve, but they do need to hustle. Demonstrate this hustle early and often on client and internal work alike. Push your idea forward while fulfilling your responsibilities, and as your idea shows promise, additional hours will be allocated.
Startup founders work more than a traditional 9-5. At least our version of foundership allows you to spring for avocado on your sandwich. What should you be hustling to accomplish?
Pre-sell that first customer. Nothing validates a concept more compellingly than the product-market fit proof from someone willing to pay real money. That first customer informs decisions of what to build—they know what they want and more importantly, what they’re willing to pay for. (Higher p).
With the first customer in hand, then comes the market research to see if this product is something folks will buy and use, who those people are, and what types of people are most numerous. (higher r).
Finally, we identify possible strategic value—is that product a potential opportunity to showcase specific technical talents? Will it help further our long-term ambitions in the BCI space? Does it add to our PR, our marketing, or is it simply fantastic copy?5
Improvement
Most startups fail. (Duh). Most low p, high r projects will fail. But the companies that attempt large numbers of these projects will be those with world-changing products and the best-lined pockets in the decades to come. We embrace failure and success alike6, learning from both. The mathematics are simple, but adhering to their implications requires more than some basic facility with algebra.
The best place to start is recognizing that some of the most promising ideas are the ones with the lowest probabilities of succeeding.
Then, just maybe, we can encourage the world to empower, rather than imperil, those willing to take those risks.
1 Other reasons include tweets from Elon Musk, viral content, and the eccentricities of billionaires, but I digress.
2 Yes, I recognize a 50% probability of losing one’s job is not the same as a 50% loss of a single year of salary, but then, VPs tend to find their way to subsequent roles despite prior failures (if CEOs of failed financial institutions circa 2008 are any indication), so I’ll just assume the 50% chance of failure implies a 50% chance of losing one year’s worth of salary and get on with my afternoon, capisce?
3 If a project’s success enriches its leader and a project’s failure damages its leader’s career, one can hardly be surprised when objectives are altered ex post facto, analytics teams are called upon to justify rather than evaluate, and skewed powerpoint presentations abound.
4 AE currently employs around 150, but we try our damndest to hire folks who would prefer a startup vibe.
5 Like, it inspires the composition of some technical-joke-filled parody?
6 Churchill once said that “success consists of going from failure to failure without loss of enthusiasm.” Perhaps, were he still alive, someone might tell him “sir, you’re a terrible javascript developer, truly horrible…” and he might have the opportunity to retort, “Well you’re a schmuck, but I can check Stack Overflow in the morning!”