Flawed psychology

February 14, 2013

Stephen Gibson acknowledges that there may be a “train wreck looming” for one subfield of social psychology but not for the field more generally (“This is bigger than social psychology”, Letters, 24 January). This subfield comprises research on social priming in which researchers use subtle environmental cues to try to unconsciously influence aspects of people’s social behaviour.

We can put Gibson’s claim to the test. Suppose we look at the current issue of one prestigious social psychology journal and select an article at random. Here is what I found when I did so. The article in question is not on social priming and was genuinely chosen at random (and is in fact the only one I’ve read from this issue). Its authors shall remain anonymous.

In the first of two reported experiments, neither of the key measures reaches the conventional level of statistical significance. Despite this, the authors report that the results provide support for their experimental hypothesis. In the second experiment, one of the key measures reaches significance but not the other, and the authors argue that their hypothesis is even more firmly supported. However, the significant result is present only among female participants. Male participants do not show the effect, the authors clearly did not make this analytic choice a priori, and they fail to report a statistical analysis across the entire sample. It would be hard to find a more flagrant example of the employment of an “experimenter degree of freedom” to extract a significant result from one’s data - precisely the sort of practice that has been highlighted in recent debates about psychology. Yet the article passed the journal’s peer-review process.

It would be comforting to think that low replicability is a problem only in social priming research, but it extends far into mainstream social psychology, as this example highlights. Of course, it doesn’t end there: false positives are no doubt being reported in all areas of psychology as well as, probably, science. But social psychologists must recognise their own acute vulnerability to poor research practices and consider how to get their house in order.

David Shanks, Head, Division of Psychology and Language Sciences, University College London

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

Assistant Recruitment - Human Resources Office

University Of Nottingham Ningbo China

Outreach Officer

Gsm London

Professorship in Geomatics

Norwegian University Of Science & Technology -ntnu

Professor of European History

Newcastle University

Head of Department

University Of Chichester
See all jobs

Most Commented

men in office with feet on desk. Vintage

Three-quarters of respondents are dissatisfied with the people running their institutions

students use laptops

Researchers say students who use computers score half a grade lower than those who write notes

Canal houses, Amsterdam, Netherlands

All three of England’s for-profit universities owned in Netherlands

sitting by statue

Institutions told they have a ‘culture of excluding postgraduates’ in wake of damning study

A face made of numbers looks over a university campus

From personalising tuition to performance management, the use of data is increasingly driving how institutions operate