Stat-spotting: A Field Guide to Identifying Dubious Data

Ever wondered what vampires and bad statistics have in common? According to Joel Best, they are both fiendishly difficult to kill. We live in a world where the media constantly bombard us with an infinite variety of facts and figures. But have you ever stopped to think about where these figures come from? Have you ever asked yourself what they really mean? Indeed, have you ever questioned whether you should be worried based on the numbers you are presented with?

If you have, Stat-spotting may be the book for you. It may also be the book for you if you have never worried about these sorts of things - because by the end of the book you will know why you really should worry.

This text aims to be a practical guide designed to help us spot some of the so-called "dubious data" that we come into contact with on a daily basis. Put another way, the text equips readers with the skills needed to make sense of, and evaluate, the numerical information that they encounter.

Topics covered include: sources - who counted and why?; definitions - what did they count?; measurements - how did they count?; packing - what are they telling us?; and debates - what if they disagree?

The text begins by providing readers with a context for some of the bigger numbers we encounter - for example, billions. This upfront approach, adopted from the very beginning, allows readers to appreciate the arguments presented and gain an appreciation of the relative size of numbers. Throughout the book, Best adopts a chatty and informal writing style that makes the text accessible and readable. The clever use of examples to reinforce the points raised in each chapter further enhances readability.

As well as discussing how numbers can be used and misused, Best also provides readers with an explanation of the underlying processes involved in research. The research process is explained in such a way that readers' attention is drawn to how studies are designed to generate numbers and how, in some cases, biases may occur in this process.

Complex statistical techniques are not presented but, rather, readers are walked through some of the underlying statistical principles that are at the heart of generating and presenting statistics. Readers' attention is also drawn to some of the fundamental ethical principles involved in the research process.

The only possible drawback of Stat-spotting is that the text is written with an American audience in mind. Consequently, many of the examples Best provides specifically draw on salient, topical issues in the US. However, readers should not let these examples put them off as they are up to date and often still topical even for non-American readers. For example, Best questions how information on crime, medical research and gender inequality is presented.

He also offers examples of how issues can be redefined to make the numbers associated with them much more impressive. We are encouraged to recognise that some of the statistics that we encounter may be biased. We are also urged to evaluate whether the research process is legitimate in such a way that readers will be left to question the very nature of the numbers that have been generated and what these numbers indeed represent.

As well as providing a light-hearted review of statistics, this book can, I think, also provide a means for readers to develop their evaluation skills more broadly as the skills needed to challenge the underlying assumptions of statistics can easily be transferred. However, a note of caution may be wise here as readers could inadvertently find themselves adopting a more critical attitude to information that they are presented with and thus generating more questions than answers.

For those readers who do want to use their new found stat-spotting skills, towards the end of the text Best helpfully provides a summary of "common signs of dubious data". These afford a practical way of transferring the concepts discussed in the field guide to the "real" world.

By the end of the text, I was in complete agreement with Best that a bad statistic is, indeed, "harder to kill than a vampire".