Citizen Science by Jamie Zvirzdin

‘Science Says So’: Sifting Evidence-Based Science from Pseudoscience

If I got a gold nugget whenever someone said, “Studies show” or “Researchers discovered,” then, a) I’d have mountains of gold, and, b) I’d give it all away for people to be more honest about their information sources and their supposedly scientific methods. We save ourselves so much time, energy, and money by knowing the difference between evidence-based science and pseudoscience, between good and bad research.

During the California Gold Rush of the 1850s, a lot of hopeful people chased the prospect of easy money. Even today you can visit sites in the West where you pay to “pan for gold” in a stream of cold, silty water. Little kids—and adults—squat by the water’s edge and use a flattened metal sieve to search for gold. A kid—or an adult—will spot something shiny in their pan and squeal with triumph, and the others will crowd around the same spot and dig furiously into the streambed.

More often than not, the squealer finds not gold but pyrite, “fool’s gold”—an iron sulfide with an entirely different chemical composition but a similar metallic luster. I’ve been a squealer before—as a kid—and I remember my utter disappointment upon realizing I’d been too quick to believe.

While the analogy of “panning for science” has as many holes as a sieve, it is true that we humans are easily fooled by false scientific claims. Hope springs eternal: If it worked for them, surely it will work for me, too, we think. But a testimonial is not science. Real scientists—and citizens who understand science—will stop and sift the gathered data very carefully before joining the ranks of eager squealers.

Unfortunately, not every scientist is honest, with others or with themselves, and sometimes bad research is passed off as good research. Andrew Wakefield falsely claimed a link between the measles, mumps, and rubella (MMR) vaccine and autism. His dishonesty, conflict of interest, and mistreatment of the 12 autistic kids in his study is a horror story of bad science. His falsified research was retracted from the medical journal, “The Lancet,” and he was disbarred from practicing medicine in Britain. Similarly, Walter Freeman, foremost advocate of brain lobotomies, kept injuring the brains of his patients even after safer options became available. In an excellent article in “The Washington Post” by Megan McArdle, she says, “In his zeal to change the world, Freeman had ended up violating the first principle of science, as laid out by the physicist Richard Feynman: ‘You must not fool yourself, and you are the easiest person to fool.’”

To illustrate the difference between the scientific method and pseudoscience, consider the following (perhaps better) analogy. Imagine you’re a detective investigating a crime, perhaps a murder at a gold quarry. Your job is to gather evidence and determine who killed Prospector Bill late last night in the cold, silty stream. The scientific method is similar to the process of gathering evidence in a criminal investigation. You start with a hypothesis, which is like a theory about who committed the crime. You then collect evidence, which is like conducting experiments and making observations. Then you analyze the evidence, which is like interpreting the results of those experiments. If the evidence supports your hypothesis, you can make a conclusion about whodunnit, about what cause is linked to what effect.

Now, imagine that instead of gathering evidence, you rely solely on hearsay, rumors and gossip to determine who committed the crime. Prospector Amos said he heard fighting down by the creek, and he saw the mayor’s daughter, Dirty Janet, down by the creek earlier that afternoon, so it must be Dirty Janet who done killed poor Prospector Bill.

This line of reasoning is like pseudoscience. Pseudoscientific beliefs rely on anecdotal evidence— testimonials—which is like hearsay, rumors and gossip. These claims are not subject to rigorous testing—in fact, there’s no way to test whether that claim is true or false, whether Prospector Amos really heard or saw what he did. Without evidence and without the ability to test the claim, it is impossible to make a valid conclusion. You, the detective, would be claiming a cause-and-effect relationship—the topic we discussed in the last “Citizen Science” article—but it is not just, not right, and not good sleuthing to believe a claim against Dirty Janet without proper evidence.

As good detectives, I invite us all to bravely investigate our own beliefs further. As a starting point, check out this Wikipedia webpage of pseudoscientific topics: https://en.wikipedia.org/wiki/List_of_topics_characterized_as_pseudoscience.

Science is an ongoing investigation of truth through time and repeated experiments by diverse experimenters. For a moment, however, let’s consider the other extreme of falsely accusing someone of murder and convicting the wrong person. It would likewise be wrong to ignore valid evidence if it were presented. If Prospector Amos has Bill’s blood on his boots and Bill’s pyrite in his pockets, it would be wrong to look the other way. Sometimes we can be so skeptical that we dismiss good research based on good evidence. It is a precious balance we must find as good detectives, and we’ll be talking more about distinguishing sources in the coming months.

Like kid-me staring glumly at a clump of pyrite in my hand, you might find yourself disappointed upon discovering you’ve been fooled. If you look at the Wikipedia list of unsubstantiated claims and realize you’ve been sold bad information by overly confident prospectors, just remember: It happens to the best of us. What matters is that we become increasingly better detectives through time as we sift fact from fiction.

As April Fool’s Day approaches, let us remember what Benjamin Franklin supposedly said—but most likely did not, since I can find no original source: “The only thing that is more expensive than education is ignorance.”

Jamie Zvirzdin researches cosmic rays with the Telescope Array Project, teaches science writing at Johns Hopkins University and is the author of “Subatomic Writing.”