How many social scientists will be good enough to qualify for the new superstar research ranking in the 2008 research assessment exercise? writes Anthea Lipsett.
The question is still being asked by assessment panels in the social science subjects as they contemplate the new categories of "quality profiles" that will be awarded to university departments: 4*, 3*, 2*, 1* and unclassified.
David Otley, the chair of Panel I, which covers business, economics and finance, said: "How good does someone have to be to be 4*? There are forms of words, but it's not quite as clear cut as it used to be. Would only Nobel prizewinners count in the 4* category?" Government-inspired selectivity of this order might prevent any academics reaching the top grade in some areas, he said.
The quality profiles more generally will mean an "enormous extra workload", he said. "Panel members will clearly have to read a higher proportion of the work than in previous exercises."
Individual researchers used to be judged to be likely to fall within certain research ratings. Now every piece of research work, from academic papers to books, has to be judged within one of the five new quality profile categories.
"It was intended to avoid the problem of people entering all their research staff regardless of how well they do, but it doesn't solve that problem," Dr Otley said. Ensuring that the same standards are applied across all subject sub-panels could prove difficult within his panel as the areas are like "chalk and cheese", he said.
Professor Otley added that his panel was keen to avoid splitting natural research groups to fit panels. "We have a robust procedure for cross-referring."
Different research outputs
His panel cites research outputs as the most important indicator of quality. Panel I expects the majority of work submitted to be articles in refereed journals, books (including chapters in them) and research monographs. The form of research submitted will be not be treated as necessarily of higher or lower quality than others - a rule that applies across the board in the social sciences.
Peter Taylor-Gooby, chairman of the social policy sub-panel, which falls under social sciences Panel J, said: "In our subject area, a great deal of work is not published in top-ranking journals but in reports to government departments, evidence to (parliamentary) select committees or similar forms of output. We will be taking all forms very seriously.
"We are fully aware that this means we will have to read everything. It involves a lot of work, but I don't think it's insuperable."
Pedagogic research
Pedagogic research fared badly in the last RAE, and input this time from the Higher Education Academy has helped to hone what constitutes pedagogic research across all panels.
People can submit pedagogic research either to the education panel or submit subject-specific pedagogic research within a discipline (for instance research into teaching and learning in physics).
The HEA was keen to ensure that researchers were aware of the differences between more localised investigative or evaluative research and the pedagogic research that fits the RAE mould.
Mike Prosser, HEA's research director, said: "Both aspects are really important, but the sector needs to draw a distinction firmly so that pedagogic research is valued by the RAE."
Reaction: 'Relieved'
The new-look research assessment exercise struck a harmonious chord with many social scientists this week, but some warned that the changes may have come too late, writes Anna Fazackerley.
Members of the Political Studies Association were delighted that the guidelines had not been drafted behind closed doors.
Justin Fisher, a member of the association's executive and head of politics and history at Brunel University, said: "By and large, we are very happy.
This is in no small part due to the fact that members of the panel have been very keen to get input from the profession. We've been very impressed."
Pedagogic researchers, who felt they had been left out in the cold during the last RAE, this week expressed relief that the guidelines acknowledged their existence much more strongly.
Helen Howard, manager of the Higher Education Academy's Centre for Sociology, Anthropology and Politics, said: "Most social scientists are going to be pleased with the guidelines - they look very useful. We've been stressing the necessity of pedagogic research for some time, and I hope the panels will interpret this advice well."
Martin Woolley, research director at Central St Martin's, University of Arts, London, said: "It's much clearer this time what will count as research and how it will be treated." Including interdisciplinary research would be much more straightforward too, he said.
But Tony Cockerill, professor of applied economics at Durham Business School, said there would still be a big divide between high-level theoretical economics research and the sort of applied research his school concentrated on. "Moderating elements have had only a weak effect in the past, and I wouldn't be surprised if that happened again in 2008," he said.
But Alan Jenkins, professor of higher education at Oxford Brookes University, warned that the guidelines had come too late to change ingrained institutional behaviour.
"The problem is that the damage has already happened. In terms of research, my view is that institutions have played very safe in terms of who they take on and what sort of research they value," he said.
He added that many institutions had turned their backs on applied research and broader interdisciplinary scholarship. "Pedagogic research will suffer again. The evidence is pretty clear that many institutions have withdrawn that sort of support," he said.
Weightings
Panel I
Economics and econometrics
- Outputs: 75 per cent
- Environment: 20 per cent
- Esteem 5 per cent
Accounting and finance; library and information management
- Outputs: 75 per cent
- Environment: 15 per cent
- Esteem: 10 per cent Business and management studies
- Outputs: 70 per cent
- Environment: 20 per cent
- Esteem: 10 per cent
Panel J
Law; politics and international studies; social work and social policy, and administration; sociology; anthropology; development studies.
- Outputs: 75 per cent
- Environment: 20 per cent
- Esteem: 5 per cent across the board
All research outputs will be treated equally, and there is no list of the relative standing of journals.
Panel K
Psychology; education; sports-related studies.
- Outputs: 70 per cent
- Environment: 20 per cent
- Esteem: 10 per cent across all sub-panels