Flawed psychology

February 14, 2013

Stephen Gibson acknowledges that there may be a “train wreck looming” for one subfield of social psychology but not for the field more generally (“This is bigger than social psychology”, Letters, 24 January). This subfield comprises research on social priming in which researchers use subtle environmental cues to try to unconsciously influence aspects of people’s social behaviour.

We can put Gibson’s claim to the test. Suppose we look at the current issue of one prestigious social psychology journal and select an article at random. Here is what I found when I did so. The article in question is not on social priming and was genuinely chosen at random (and is in fact the only one I’ve read from this issue). Its authors shall remain anonymous.

In the first of two reported experiments, neither of the key measures reaches the conventional level of statistical significance. Despite this, the authors report that the results provide support for their experimental hypothesis. In the second experiment, one of the key measures reaches significance but not the other, and the authors argue that their hypothesis is even more firmly supported. However, the significant result is present only among female participants. Male participants do not show the effect, the authors clearly did not make this analytic choice a priori, and they fail to report a statistical analysis across the entire sample. It would be hard to find a more flagrant example of the employment of an “experimenter degree of freedom” to extract a significant result from one’s data - precisely the sort of practice that has been highlighted in recent debates about psychology. Yet the article passed the journal’s peer-review process.

It would be comforting to think that low replicability is a problem only in social priming research, but it extends far into mainstream social psychology, as this example highlights. Of course, it doesn’t end there: false positives are no doubt being reported in all areas of psychology as well as, probably, science. But social psychologists must recognise their own acute vulnerability to poor research practices and consider how to get their house in order.

David Shanks, Head, Division of Psychology and Language Sciences, University College London

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

Female professor

New data show proportion of professors who are women has declined at some institutions

celebrate, cheer, tef results

Emilie Murphy calls on those who challenged the teaching excellence framework methodology in the past to stop sharing their university ratings with pride