The REF is wrong: books are not inferior to papers

Monographs typically constitute a scholar’s greatest achievement, but REF strategists discourage their production, says Bruce Macfarlane

十二月 7, 2017
Eleanor Shakespeare illustration (7 December 2017)
Source: Eleanor Shakespeare

With the rules now finalised, UK universities are busily making preparations for the 2021 research excellence framework. Mock-REF exercises are taking place across the country, passing judgement on the quality of researchers’ work.

One of the now familiar mantras of those tasked with REF strategy, such as associate deans for research and departmental coordinators, is that books are essentially inferior to journal papers, or simply “don’t count”. Their only question is: “What papers are you putting forward?” If you suggest to them that one of your best “outputs” is a book, they look at you with a mix of pity and scorn.

In theory, the REF does not differentiate between types of outputs, so books count the same as journal papers. This seems the very least one would expect given that writing a 60,000- to 80,000-word book can involve considerable scholarly time and effort. Indeed, the 2014 REF permitted requests for books to be double-weighted in assessment. However, in many subject areas, relatively few such requests were made; in many humanities and social science disciplines, such as history and geography, the submission of books and book chapters actually fell in 2014, compared with the 2008 research assessment exercise.

The first problem is that a “book” comes in many different forms, carrying implications about relative status. Student textbooks or chapters in lightly edited collections from minor academic conferences are bottom of the pile, while single-authored academic monographs with prestigious publishers sit at the top. But even the last, often the result of many years of work, can be unfavourably looked upon.

One reason is that the humanities and social sciences are increasingly expected to adopt the cultural norms of the hard sciences, whose latest findings are published in journals. Critics claim that books don’t contain sufficient originality and replay previously published work. But much the same accusation could be levelled against many journal papers. Another argument is that books are not subject to peer review in the same way as papers. But there is an exaggerated faith in the integrity of peer review. In fact, it tends to reinforce rather than challenge disciplinary norms and is not immune from academic cronyism. Besides, any reputable book publisher will ask for a proposal and seek out suitable reviewers. Getting a contract with a good publishing house is as competitive a business as any other aspect of academic life.

There is an important intellectual case for people to write books, too. A few thousand words might be suitable for reporting data from an empirical study, but it is not enough to develop a deeper conceptual argument. A good academic monograph is much more than a synthesis of the taken-for-granted or some form of outmoded personal indulgence. Being able to sustain an argument based on theoretical and/or empirical resources over several hundred pages is no easy task. Carrying it off successfully is an important indicator that a person has genuine intellectual capability. Most importantly, academic monographs often play an important role in putting forward fresh and controversial perspectives.

The major figures in the humanities and social sciences have always written books and most see them as their major works. The citations to my own books are modest by comparison but outstrip those to most of my journal papers and I also regard them as my main intellectual achievements.

All the talk now is about impact but this is an area where books can do better than a single journal paper. A book is likely to reach a wider audience than a paper in a specialist journal read only by other academics. Books attract reviews that critically assess their value, and there are other measures of their influence, such as sales and invitations to speak about them.

I am concerned that spreading a negative message about books in the REF is having a damaging effect on newer researchers. I recently completed a study analysing the autobiographical profiles of three generations of scholars in the higher education research field. Those who started their careers in the 1960s tended to publish at least as many, if not more, books, book chapters and reports as journal papers. By contrast, the most recent generation have long lists of journal papers in their CVs and not much else.

As long as the REF survives in its current form, this might be a sensible strategy. However, we need to remember that the REF is a parochial British exercise based on blind faith in peer review. If UK-based researchers ever want to work overseas, what counts is to have publications that are well cited and appear in a good journal or with a respected publisher. Employers will be interested in your h-index, not your contribution to the REF.

The people dismissing the value of books tend to have one thing in common: they have never written one. Perhaps they should go away for a year or two and try.

Bruce Macfarlane is head of the School of Education at the University of Bristol. His latest book, Freedom to Learn, is published by Routledge.


Reader's comments (4)

Actually, it is largely the H and not the SS that publish books, as any publication database can show you. Books suffer from a lack of definition as to what the object covers and also a vast number of pay-to-publish outfits that print on demand, and the demand is sometimes very small. As long as books are necessary for promotion, as in many countries not so far from the UK where never mind the quality feel the width approach is adopted, Mickey Mouse publishing will continue, and it is not only the UK that will be dragged down. Part of the problem lies in quality assessment. Journal standards and their peer review methods are known, and give a good proxy of quality. Books are another kettle of fish. Books are often the highest level of achievement,and this must be praised and defended, but low achievement books are rife. The analysis of evaluation protocols and their strengths and weaknesses is being actively undertaken within the ENRESSH COST network. We have SIG looking at books both from a bibliometrics and non-bibliometric aspect. Many of us are in fact humanities scholars keen to promote good quality outputs. Evaluation and books are a Europe-wide question, and beyond as we have observers from outside the hallowed EU. Unfortunately we have little UK input, it is as if you have already left Europe and have confused Europe and European values with those of the EC.
But in the last REF, across the humanities (panel D), those books that people did put in did extremely well. They were much more likely to score 4* than articles. That was true for the ones that were double weighted too. The take-home message was 'the book is back'. If people haven't realised this, the problem is not due to the REF, but to false beliefs about the REF.
In 2014 I saw no evidence of books being undervalued in sub-panel 34 Art and Design: History, Practice and Theory. Some sub-panels may choose to (mis)interpret the messages of the REF, but it seems unfair to suggest that the REF itself is flawed in this respect. Main panel D reported that "almost all the requests for double-weighting – 98 per cent – were accepted." (REF 2014: Overview report by Main Panel D and Sub-panels 27 to 36, January 2015).
There are high and low quality book publishers - just as there are high and low quality journals. What matters is not the medium, length or format but the standards. It is not brain surgery to distinguish: University presses always have a strict review process before and after the m/s is completed. Trade presses do not always follow the same process. A university press book is usually worth 100 articles in terms of impact on the discipline and citations. Use the evidence of citations to evaluate.


Log in or register to post comments