Letters

Pros and cons of the metrics system

The main problem with the research excellence framework is the huge amount of time and money that it consumes, distracting academics from scholarly activities and diverting funds to employ squads of administrators to manage it (“For richer, for poorer”, 9 May).

In a recent blog post, I reported an analysis I did showing that you could pretty accurately predict the research assessment exercise’s psychology results by taking an H-index (indexes that attempt to measure the productivity and impact of academics’ published work) based on departmental addresses.

In the comments on the post, there is further evidence that this is also true for physics. Computing the H-index takes an individual two to three hours, as opposed to the two to three years spent by legions of people working on the REF.

At present, we don’t know whether such differences as exist between H- index ratings and RAE panel ratings mean that the latter were better. For both psychology and physics, once you had taken the H-index into account, additional variance in funding levels could be explained by whether there were departmental representatives on the assessment panels.

This suggests that the additional value given by having expert assessment incorporated in such evaluations may just be adding subjective bias, which does not necessarily indicate any malpractice but could reflect the advantage that panel members have of intimate knowledge of how the panels work.

Most of us don’t like metrics, and I accept that they may not work in the humanities, but I would suggest that if we are not going to use a metric like this, we need to do an analysis to justify the additional cost incurred by any alternative procedure. If the additional cost is very high, then we might decide that it is preferable to use a metric-based system, warts and all, and to divide the money saved between all institutions.

Dorothy Bishop
Via timeshighereducation.co.uk

One of the problems with the use of citations as a measure of research quality is that the method assumes that the higher the number, the greater the quality. Ignoring the possibility of “tit for tat” reciprocity between mates, what if your article is cited and immediately preceded by “for a total misunderstanding of even these basics, see Mead…”?

In addition, citations don’t work as a measure of anything where the chance of others quoting your work is low: if I were ploughing a relatively lonely research furrow, I’d prefer to take my chances with a subpanel of the great and the good. The fact that no one else is relying on my work because no one else is interested in it (or has even heard of it) says nothing about whether it is good, bad or indifferent. Like impact, citations therefore have a tendency to skew personal research interests into institutional research agendas, favouring more of the greatly populated same, not those who are pushing boundaries and exploring for its own sake.

David Mead
Professor of public law and UK human rights
University of Essex

Right royal appointment

It is regrettable that a publication written mainly for academics should choose to cover The Sunday Times’ version of the facts about the election of the Duke of York as a royal fellow of the Royal Society (The week in higher education, 9 May). Eighty-five per cent of those who voted supported the election, and while the ballot paper did only have a box to vote “yes” to confirm the nomination of the society’s council - the elected representatives of the fellowship - explicit instructions were also sent to every fellow on how to register a “no” vote.

Your reporting of The Sunday Times article made reference to the fact that only 11 per cent of fellows supported the nomination, but neglected to mention that only 1.8 per cent opposed it. As is common with democratic elections, many of the electorate chose not to vote, but it is ridiculous to use this fact to speculate on their intentions.

Prince Andrew was nominated for the distinct category of royal fellow on the basis of his support for British science (and young scientists in particular) and for his work to improve the links between academic researchers and industry. These areas are vital if we are to secure the long-term success of science in the UK.

Martyn Poliakoff
Vice-president
The Royal Society

All bark, no bite?

The recent Department for Business, Innovation and Skills’ draft guidance Alternative Providers: Specific Course Designation states on page 9: “The QAA safeguards academic standards and quality in UK universities and colleges, so that students have the best possible learning experience.” Unfortunately this statement is incorrect, for at least two reasons.

First, it is only academic staff, working within established professional values, mechanisms and practices, who can actually safeguard academic standards and quality.

Second, the Quality Assurance Agency itself does not have the necessary powers or sanctions. Under UK law, British universities remain answerable only to themselves for their academic programmes and qualifications. While the QAA can certainly draw attention to any case where it considers that standards or quality may be at risk, it cannot require an institution to change its programmes or qualifications.

We are aware of three qualifications to this statement.

First, where a private provider has obtained degree-awarding powers, identified risks to standards or quality could be grounds for the Privy Council’s declining to renew those powers on the advice of the secretary of state.

Second, where the offending institution receives cash from the Higher Education Funding Council for England, Hefce may withdraw the money, according to Policy for Addressing Unsatisfactory Quality in Institutions, a document first published in 2009. However, this policy has never been invoked. In any case, many institutions will soon be receiving little or even no Hefce funding. And the funding council’s vires for taking such action - and indeed action in the field of quality more generally - are questionable because the 1992 Further and Higher Education Act expressly excludes it from the scrutiny of academic standards.

Third, the UK Border Agency (and whatever replaces it) may withdraw the status of highly trusted sponsor in cases where the QAA advises that standards or quality may be at risk. It therefore appears that the only clear remedies that are available where a provider takes serious risks with quality or standards are those operated under the aegis of the home secretary.

Roger Brown
Professor of higher education policy
Liverpool Hope University

Geoffrey Alderman
Michael Gross professor of politics
University of Buckingham

Limited choice

Steve West’s contention that students need wide-ranging information to make their choice of university and course would be welcome if he were not simultaneously undermining it (“Working class law suits”, Letters, 2 May).

He approvingly cites the National Student Survey satisfaction score of 91 per cent recorded for the University of the West of England’s law programme. This might lead students to think that UWE would support such a well-regarded course, and that popular courses at the institution are both valued and safe from closure. Yet the politics and international relations programme West tried to close in February (before performing a public U- turn) scored substantially higher than law in the NSS: international relations achieved 95 per cent and politics a perfect 100.

Data such as the NSS figures are not the only things students consider. Increasingly, many look deeper into how universities are being managed, and decoding institutional “mission statements” is now essential. The clear message at some post-1992 universities is that “academic” subjects are best left to the Russell Group, and that post-92s are somehow leading the charge against elitism. Students may well have more data with which to make choices, but those choices are stymied by the shrinking of academic disciplines outside a favoured elite. West’s recent actions suggest that he does not think students should be making “academic” choices at his university. Nothing is more elitist than limiting the options of students who do not achieve AAA grades at A level and places at Russell Group institutions.

West’s complaint that “we need to stop focusing on some imagined idea about the ‘best’ universities” is disingenuous. “Best” universities would not threaten popular and effective courses or cut the breadth of academic offerings on the grounds that the Russell Group also provides them. Nor would they charge Russell Group fees without Russell Group opportunities.

Tony King
Bristol

Advance warning

Pauline McGovern makes a good point about vivas (“Harsh interrogation”, Letters, 2 May). Their main purpose is to establish that candidates understand their work and can explain it in the context of the literature. Catching them off guard with a surprise line of questioning in an already tense atmosphere does not help.

I have led doctoral vivas in several countries, and it is my belief that the UK system is best fit for purpose. However, I would suggest that one rule of the European system be adopted, namely that of advance written notice by the external examiner of major issues to be raised at the viva (with a chair appointed to ensure that undisclosed questions are ruled out).

Advance knowledge of the examiner’s line of questioning does not make the process any less rigorous - the candidate may still fail if they cannot defend the thesis - but it does reduce the polarity of power in the room between examiner and student, and introduces an element of fairness if the latter at least knows where the attack is coming from.

David Jessop
Honorary senior lecturer
University of Bristol

Turf flaws

While John Furlong may have right on his side in his criticism of some aspects of recent English education policy, he is making the classic mistake of trying to persuade people that the grass is greener elsewhere when he says that “in the Republic of Ireland, postgraduate training for teachers is being lengthened from one to two years…and in the future all courses are to be provided by research-led university departments rather than stand-alone teachers’ colleges” (“In pursuit of the truth”, 2 May).

The former is true, although whether it will have the desired effect of “improving standards” (whatever that means exactly) remains to be seen, as it is a policy devised by a regulatory body, the Teaching Council, that has little or no serious interest in intellectualism. The latter development is the product of a less-than-coherent report (authored by Pasi Sahlberg, Pamela Munn and one John Furlong) that has occasioned unnecessary confusion, antagonism and opportunism.

The grass may be green(er) in Ireland, but that is largely because it rains so much here.

David Limond, Andrew Loxley and Aidan Seery
Trinity College Dublin

Rearing window

In his review of Peter Mandler’s Return from the Natives: How Margaret Mead Won the Second World War and Lost the Cold War, Chris Knight characterises as “nonsense” Mead’s belief that the swaddling of Russian infants led to adult Russians seeking “maximum total gratifications” (“The rise and fall of a mother”, Books, 2 May). Yet this is an idea open to empirical test.

The classic study of British adults by Frieda Goldman-Eisler (1951) showed that early weaning and its combination with impulsiveness-aggression might prompt the development of oral pessimism. So I would expect Russian child- rearing practices, especially swaddling, to have a measurable influence on the development of personality.

R.E. Rawles
Honorary research fellow in psychology
University College London