I’m about to insult you.
It’s for a good cause, though, I promise.
The Cargo Cult
The great physicist Richard Feynman gave a commencement address at Caltech in 1974 entitled “Cargo Cult Science.” You should read it. But if you don’t, here’s a poor summary: during the second World War, the previously isolated Melanesian Islanders found themselves in the midst of a logistical war as the Japanese and then the Allies airdropped vast quantities of cargo on the islands to support their advancing troops. The islanders had never before seen such a bounty of goods, but eventually the war came to an end and the shipments stopped.
The islanders, hoping to revive the flow of goods, developed so-called “cargo cults”: religious groups that painstakingly replicated the rituals they had seen Japanese and U.S. ground crews use to “summon” planes, fashioning headphones, runways, antennas, and even mock planes from local materials.
To quote Feynmann:
“They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.”
Science has produced miracles. The application of its principles by a pantheon of brilliant minds has produced wonders, elevated human existence, and spared billions from untold suffering.
Piles of mediocre, rote, dogmatic work claim that same title for themselves and say that simply because they share a noun — “science” — with the pinnacles of human achievement they deserve its respect and societal deference. More often than not, they get it.
Why should they?
The Cocktail Party
The cocktail of hubris, authority, and a near-total lack of external safeguards is a heady one, and I don’t blame anyone for finding themselves a bit tipsy after drinking it.
Recipe: Institutional Inertia Spritzer
(Serves 12. Prep time: since the 60’s, more-or-less.)
- Hubris: We used statistics and advanced computational methods — how could it be wrong? We learned this. I have a PhD.
- Authority: Science split the atom. Science understood the cosmos. Science cured polio, measles, and smallpox. Why don’t you believe social science? Or the nutritional scientists who insist that kale cures cancer? And so on…
- Lack of External Safeguards: The people with the closest proximity to the work — those doing it, those supervising it, and those citing it — have the greatest incentives to suppress or ignore its failings. Reviewers are overworked and prone to being quite trusting. In an era of increasing specialization, they also have powerful incentives to avoid critiquing central methods or dogmas of their sub-fields, lest the critique affect the validity of their own work.
Serve on the rocks, and shaken, not stirred.
Straight-up scientific fraud is alarmingly common — see Retraction Watch — but the true issue is the enormous quantity of cargo-cult science done by people with decent intentions who lack the “utter honesty” Feynmann describes: they are willing to fool themselves, and so fool everyone else.
This bad work does not sit untouched and unread. It does not simply await refutation — there is too much of it to refute anyway. Rather, marked by government funds, institutional approval, and the ponderous authority of jargon, it fuels a self-perpetuating cycle of bad work.
Eventually, this spills into the real world. Policy is made; articles are written in Yahoo News about kale. Prescriptions are given or withheld. Faith is proclaimed in science, but credence is given to the bureaucracy that bears its name rather than its principles.
Psychology, for example, has been hosting a helluva cocktail party for decades now, leading to what has been termed a “reproducibility crisis”. A series of increasingly robust and comprehensive replication metastudies have scandalously established that only about half of the published psychological literature, including many classic papers, can be replicated.
“Science is a horizon to search for, not a prize to hold in your hand. Also, I miss getting my tummy tickled.”
— Waddles the Pig, Gravity Falls
(Voiced by astrophysicist Neil deGrasse Tyson, who we’re all pretty sure wrote this line — or the first half, anyway.)
“All models are wrong but some are useful.”
— Often attributed to statistician George Box
Science Should Not Be a Religion
Science is not a Bible of Truths and scientists should not be priests. One cannot meaningfully “believe in science”: that is a religious position and social signaling. Rather, believe in the axioms of the scientific method and from there derive confidence in its replicated, battle-tested results.
This Storybook is an Allegory
In this storybook, I plan to illustrate, in a simplified and stylized way, the realities, culture, and common failings of the modern scientific endeavor. I chose astrophysics because every culture since the dawn of history has been talking about the night sky; atmospheric science, statistical psychology, econometrics, and theoretical sociology not so much.
The stories will use ancient explanations of the night sky to explore the modern scientific process of investigating the cosmos, but they are really meant to be read as allegories for the entire modern scientific endeavor.
(And I do have some idea of what I’m talking about: I’m drawing from my own experiences working in at least five different research labs in as many distinct fields.)
An Apology About Astrophysics
I did promise an apology, so here it is: I am not a physicist; at most I am a half-baked mathematician. (And unlike steaks we are not at our best medium-well.) Astrophysics doesn’t deserve what I’m doing to it here, but it’s the only field I can do this with.
Physics is by far the purest of the sciences and its practitioners have an extraordinary record of confronting the enigma of Nature with open minds, brilliant solutions, deep insight, honesty, and self-reflection.