The Best Idea Is Also The Most Popular Idea, And Often Not The Best Path
Think of a problem whose solution would generate an immense impact in our world. Perhaps that problem is political, financial, or scientific. Perhaps you are pondering tribalism and disinformation or the ramification of rising interest rates on the solvency of banks, or finding a grand unified theory, or solving AGI alignment. Reams of scholarly literature, pontification on the internet, and countless nascent ventures have certainly been devoted to solving whatever problem plaguing society popped into your mind.
Now think of the three most promising approaches for addressing that problem. Got ‘em in mind? Fantastic.
Toss them aside. Then choose the path forward.
Course Correction
The most popular ideas are often popular for a reason. They demonstrated promise, produced superior results when compared with their predecessors, and chipped away at the thorniest challenges of their age. They are well-regarded, and as a result, also awfully well-tended.
It is this process that impedes progress, not through nefarious collusion nor the resistance of Luddites. It is simply that revolutions, scientific or otherwise, typically require voices from beyond the siloed ivory towers of their fields.1
Regardless of the field or the topic, rigid adherence to one particular flavor of intellectual reasoning is bound to encounter some limitations.
As a thought experiment, imagine that the fate of all humanity is tied to a particular piece of software currently running through an infinite loop. While it is in operation, we all live. When it stops, we all perish. The greatest thinkers assemble to address the matter and questions of great import are posed. The first question is simply “for how long will this program run?”
A physicist notes that in the absence of solar energy, the surrounding environs will become so bitterly cold that all molecular activity will cease, ending the operation of the “society sustaining software” (call it S3). The physicist calculates that we have approximately 5 billion years until this occurs.
Next, a computer scientist studies the underlying source code, notes that while the infinite loop will not terminate, the loop itself was called by some higher order function called the universal software administrator (call it USA) and this particular operating system is likely to endure for far less than 5 billion years, given risks from viruses and other malware. They estimate the lifespan of the current program at a few decades, not a few eons.
Finally, an economist steps to the microphone, notes that the software is running on a server located in a office park in an exurb of Chicago whose facilities manager has a proclivity for off track betting on horse races and unbeknownst to humanity, has mocked the losing wager of a fellow degenerate who also happens to be a well-connected member of the corrupt political class of Illinois who is likely to pass a law raising the rates for electricity borne by office parks within a one-block radius in which this server happens to reside. The facility’s coffers will reach $0 in approximately seven months, after which all human beings will perish.
While the reasoning of the physicist, the computer scientist, and the economist are all sound, rigid adherence to any of those particular disciplines can omit crucial risk factors worthy of consideration.
The Fourth-Best Idea
Now, given that we already have the approach of the physicist, the computer scientist, and the economist in our back pockets, what happens if I demand that you consider the risks to the society sustaining software from the perspective of additional disciplines?
Perhaps a meteorologist will note that the building housing the server is likely to incur a lightning strike and requires a more robust surge protector? Perhaps a structural engineer will tell you that the beams in the roof are likely to buckle beneath the weight of snow in an atypically rough Chicago winter? Perhaps an entomologist will discover that some termites will soon eat through one of those beams and cause a collapse long before the snowfall will? What if an etymologist warns that a verbose screed written by a Chicago-based essayist will be read by the aforementioned facilities manager who, in a fit of nihilist, suicidal rage at having been outed for his gambling addiction, levels the facility and its servers?
X-Risk
The tongue-in-cheek tone notwithstanding, from the millions of hours and dollars devoted to string theory or the insularity of any number of intellectual or activist communities, problems remain unsolved in large part due to the infinite, unrelenting, rigid pursuit of a single path within a single discipline.
The ultimate solution to Malthusian concerns in the 60s did not lie with governmental policies in favor of sterilization and other reproductive deterrents2, but rather, technological advances in food production. The conservation biologist offered one set of tools and at some point, the precision agriculture experts offered their own.
Among the most important elements of course correction is the intellectual humility required to accept that there are likely far more available potential courses than we are currently considering.
The problem with the best idea is not that it is necessarily the wrong approach, but rather, that it is almost certainly the most popular.
Existential challenges are difficult (by definition, or they’d be solved already). This means most (if not all) previous avenues of inquiry have already failed. This also means that our prior about the current “best idea” should recognize that even this, the most promising approach, is also likely to fail.
It follows therefore that if the problem is to be solved, it is more likely to be solved by some approach other than the current, best idea.
Thus, on behalf of all not-the-best-ideas in heads and texts across the planet, let’s force ourselves, at least some of the time, to eschew what consensus deems best, and pursue another, severely neglected approach.
1 Thomas Kuhn gives this idea an extensive treatment in “The Structure of Scientific Revolutions,” which has aged extremely well in the six decades since its publication.
2 Paul Ehrlich’s “The Population Bomb” did not age nearly as well as Kuhn’s, given roughly the same number of years since their publications.