Teaching students how to spot scientific misinformation

The information revolution through which we are living has benefited society in many ways. More people than ever before can easily access numerous sources of expertise. Yet that information revolution has also made it exponentially easier to fabricate vast amounts of misinformation. This is particularly true in the case of scientific information.

Misuses and abuses of scientific information have generated some of the most well-known misinformation campaigns in recent years. Prominent examples include the anti-vaccination movement, climate change denialism, the return of race science, specious forms of alternative medicine, and election conspiracies.

Learning to identify misuses and abuses of scientific information is an important part of contemporary education. This article provides some essential recommendations for understanding where it comes from and how to guide students to counteract it.

Headlines

Most members of the public do not encounter scientific information in an unmediated fashion. Rather, it comes to many people through headlines and second- or third-hand reports. Such filters add layers of interpretation to scientific data, often selectively altering its meaning in minor or major ways.

For example, media outlets commonly overstate the significance of public opinion surveys. Consider the case of political polls in election cycles. Journalists habitually go beyond reporting the results of individual surveys based on relatively small numbers of respondents to announcing, speciously, that they reveal how large demographic groups think and feel in uniform ways. Political media relies on a vast industry of privately funded opinion polls, which vary wildly in quality and rigour, to generate revenue-generating storylines about which candidate is supposedly ahead or what issue allegedly animates different voting blocs.

It’s important to remind students that, in scientific terms, individual opinion surveys are always based on relatively small samples of a general population. Isolated polls indicate what different groups might think or feel about a specific set of topics; they do not provide direct or conclusive proof that all members of those groups think and feel that way, however. Moreover, asking likely voters and first-time voters, or low-information and high-information voters, the same questions may yield significantly different results. Individual survey results become more scientifically useful when one tries to replicate those results over time while comparing them to other sources of empirically valid information.

Encourage students to protect themselves against scientific misinformation by evaluating whether headlines accurately report the results of opinion polls. When faced with news items that tell seemingly comprehensive stories about what the public writ large allegedly believes, ensure that students determine how many people (or how few) responded to a given survey. Encourage them to examine whether respondents are representative of larger social classes. Ask whether a news item over-generalises the significance of respondents’ answers to a specific survey by making it tell a simpler and more compact story than the data supports.

Language and data

The issue of how media sources report the results of scientific studies highlights a deeper issue: how irresponsible word choices can promote misuses and abuses of scientific information. Scientific research is always interpretive in nature, even when it deals with quantitative results. The meaning of data is not self-evident; it requires interpretation and judgement calls. This is where human error can slip in.

Some of the most common examples of poor interpretive work with scientific data involve attributions of cause-and-effect relationships between phenomena when causal links haven’t been proven. For example, claiming that vaccines cause autism, that a new economic policy caused a crash in the stock market or that a specific form of mass media caused a narcissism epidemic in teenagers mistakes broad correlations among miscellaneous data points for a causal relationship.

This confusion of correlation with causation also assumes a simplistic linear sequence. Just because something happened before another event doesn’t mean that the former caused the latter. Misinterpretations of this sort not only promote scientific falsehoods; they also make determining the true origins of widespread problems or posing effective solutions to them more difficult.

• Collection: Teaching critical thinking
• Teach research methodology using these low-prep exercises
• How to embed critical thinking from course design to assessment

Language can also corrupt scientific data in the case of poorly worded survey questions. Questions that inform respondents about a political issue from an obviously skewed perspective before asking their views on that issue shape opinions rather than measuring them. Questions that ask respondents to register their feelings about a list of many different topics or events, rather than one isolated issue, can confuse respondents and make it easier for researchers to cherry-pick their responses. Questions that unfold in a sequence designed to raise respondents’ concerns, anxiety or negative thinking about a topic risk priming those respondents to submit preferred responses. Even responsibly worded survey questions require a minimal level of interpretive work from researchers, who must seek to understand what it means for a particular percentage of respondents to express one kind of opinion versus another on complex social, political, economic or moral issues.

Looking closely at the language choices in scientific research is another way that students can guard against the harms of scientific misinformation. Encourage them to notice whether examples of survey research use the faulty language choices described above. Doing so can help them judge whether those language choices have produced either credible or specious scientific data.  

Methodologies

Responsible scientific researchers include a clear explanation of their research methodologies in published studies. Doing so fulfils an important ethical obligation. Methodological statements provide members of the public, including non-scientists, with tools to judge for themselves whether the results of a study are valid as reported.

The methods section of scientific research should give readers an introductory explanation of who, what, when, where and why questions. How many researchers and/or participants were involved? What specific research protocols were used and under what conditions? Why did researchers use those protocols under such conditions, or why did they make some methodological choices and not others?

Remind students that no scientific study of natural, medical or social phenomena is flawless and comprehensive. An effective statement of research methods should provide fellow researchers and members of the public some means to assess for themselves the scientific strengths and potential shortcomings of individual studies.

By the same token, statements of research methodology that only vaguely explain how data was collected and analysed often suggest irresponsible uses of scientific information. Highlight that methods sections with confusing or unclear explanations about who was involved, how researchers interpreted statistical information, or what expectations they imposed on such data are all common red flags for potential misuses of scientific evidence.

For this reason, encourage students to always read closely the methodology of scientific research to assess its integrity. A confusing, vague, or poorly written methods section is usually a warning sign of serious underlying issues with a given study.

This article has summarised some of the most common misuses of scientific evidence. Learning how to distinguish sound from specious scientific claims is a critical academic as well as civic skill in our current information age. 

Bradford Vivian is professor of communication arts and sciences and director of undergraduate studies at PennState College of the Liberal Arts.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.

Campus Misinformation: The Real Threat to Free Speech in American Higher Education