Cargo Cult Science: A Muddled Term

The term “cargo cult science” gets bandied around a bit (e.g. Forbes here, Goldacre over here). The ohrase was coined by the charismatic scientist Richard Feynman. However, the speech in which Feynman introduced the term is muddled, and the whole cargo cult tale is a misleading metaphor for what Feynman went on to describe, as I’ll explain in this post.

The Cargo Cult

Cargo cults are a strange phenomenon that sprung up on some Pacific islands over the last 150 years or so. Over to Feynman:

In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to imitate things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas–he’s the controller–and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land.

The cargo cult itself is a wonderful, tragic, captivating tale — which explains why people remember it so well. And there clearly is a metaphor in there: these people are following the same motions as others (experts?), but lack the understanding of what was actually going on and so are ineffective.

Now, being a cargo cultist is quite an insidious accusation. Let’s say you’re a teacher, and I accused you of being a “cargo cult teacher”: you follow all the apparent actions of being a teacher, turning up to class, standing up to talk to the kids, setting them exercises, marking their homework, but you’re not really teaching them anything at all. Ouch! How do you defend yourself against that? (I guess you would probably look at your students’ exam marks.)

Cargo Cult Science

Anyway, back to cargo cult science. Feynman follows this metaphor:

I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential

So, how do we identify this cargo cult science? It’s all very well to say that “it looks like science but misses something essential”. We need to be able to distinguish between “looks like science” and “actually is science”. Feynman goes on:

Now it behooves me, of course, to tell you what they’re missing. But it would be just about as difficult to explain to the South Sea Islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones.

Great! So cargo cult science lacks something essential, but it’s too difficult for Feynman — the master of scientific explanation — to explain it. That doesn’t sound very promising. However, he does go on to say:

But there is one feature I notice that is generally missing in cargo cult science…It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid–not only what you think is right about
it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked–to make sure the other fellow can tell they have been eliminated.

So cargo cult science is science which is not transparent, and which does not explain its potential limitations. But what does that have to do with the cargo cult analogy? The cargo cultists do not lack transparency, their limitation is not insufficient explanation of their own methods. Summoning cargo is not a good metaphor for carrying out scientific experiments.

The whole cargo cult tale is completely irrelevant to what Feynman went on to talk about, which is science that lacks integrity and transparency. These are valid problems, but I think we should just refer to these problems as bad science or dishonest science — the cargo cult metaphor has nothing to do with it. Feynman used an ill-fitting metaphor to capture interest in a speech, and somehow it’s become one of the most famous things that he said.

(If you want to know why Feynman is so great, he tells you himself in the book “Surely You’re Joking, Mr Feynman?”. It’s a good read though, and it includes the above speech as an epilogue.)

Advertisements

2 thoughts on “Cargo Cult Science: A Muddled Term

  1. Bad science would be setting up an experiment with 30 variables, and changing them arbitrarily until your conclusion approaches your hypothesis successfully and then publishing it as a scientific discovery.

    Cargo cult science looks so similar to an honest scientific experiment, as to be easy to confuse. Consider the following inquiry:

    Why, when I drop a feather and a hammer from an equal height above a flat surface does the hammer hit first first?

    Hypothesis:

    The hammer weighs more than the feather.

    Experiment:

    Increase the mass of the feather by tying a slug of iron equal to the mass of the hammer minus the feather’s mass to it.

    Result:

    Modified feather and Hammer now fall at the same speed.

    Conclusion:

    Mass affects which objects reach a uniform surface first given an equal starting height.

    We’ve done science! Of course, this is bad science, but it’s not obvious at first blush, and if you didn’t know any better and lacked scientific integrity or rigor, you, or an unwitting public, would be prepared to believe that mass affects rate of decent when an object is subjected to gravity.

    We’ve dressed ourselves up as scientists, created a hypothesis and an experiment that confirms our hypothesis. But something’s not quite right. As Feynman notes at the end of his lecture, we’re adjusting the length of the antenna on the wooden headset to see if we can improve reception.

    As to the accusation of practicing cargo cult teaching being harsh, you better believe it is. But the reality today is that there are teachers doing the equivalent of setting their room in order, standing in front of the class and doing what the person before them did and wondering why kids aren’t learning anything. But the data generated by standardized tests and teacher evaluations is not scientific, and I would be hard pressed for how you scientifically test organic beings that can wake up in a bad mood or forget to eat breakfast. Sure quantity of testing will help to smooth it out, but there’s simply so much variability that standardized tests are almost useless. And most educators are the first to point this out when the tests suck. When they’re good, of course, they’re lauded for doing a good job. This is cargo cult science. This is the natives setting up an academic curriculum out of wood and wondering we don’t get any consistent results (some kids flourish, many flounder).

  2. I agree that something is wrong with Feynmans speech. There is no strong link between the “Cargo Cult Tale” and his explanations following. However, the “Cargo Cult Tale” is so famous, because of its iilluminating truth in it. Todays institutes are full of cargo cult scientists and in fact, they do not know it! Perhaps we all are cargo cult scientists in some aspects of our work and life. I think that the “Cargo Cult Tale” is far from being irrelevant. From my point of view, this is not a problem of integrity, its more a problem of bad knowledge. An experiment, which was good science 100 years ago, would be cargo science today, ignoring all the knowledge of textbooks and scientific papers during that time. There is strong evidence that most scientists today are mainly interested in publishing more papers, but not reading them, not to mention “understanding”. And this was the message of Feynman’s speech from my point of view: “Try hard to understand, what you are doing. Do not rely on formalities.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s