RAE 2008: Jury has designs on fruits of labour (physical sciences & engineering)

Applied research will receive much greater recognition among the panels for physical sciences and engineering in the 2008 research assessment exercise, writes Caroline Davis.

Universities will be encouraged to enter patents, software, technical reports to industry and even spin-off companies.

Ann Dowling, chairwoman of the engineering Panel G and professor of mechanical engineering at Cambridge University, said: "We've given a lot of thought to how to capture more applied outputs. We have tried to make sure excellent research in all forms receives the credit it should." She said panels also wanted to capture research into teaching methodology.

University departments will be asked to explain why the outputs they submit are their best. "We want them to tell us about what followed on - new avenues of research, research being picked up by industry. It's not a question of where something is published but what impact it has had."

Because applied research is industry sensitive, it is not published. The engineering panels hope to overcome this obstacle by accepting confidential reports submitted to industrial collaborators.

"We have many industry members on the sub-panel," Professor Dowling said.

"In addition, we're asking people to give names of industry contacts so the RAE can test the truth of statements that people have made."

Professor Dowling has discussed comparability with other the heads of the other panels. She also plans to run benchmarking exercises in which submissions will be assessed by more than one panel where appropriate.

The engineering sub-panels' criteria documents are almost identical, although metallurgy and materials has slightly different advice on the esteem and environment elements of the assessment: "When it comes to esteem we're looking at research funding and also exemplars of collaboration with industry. Under environment, there are lots of things that we think a good research environment should have. Mechanisms to transfer knowledge into practitioners is one of them, as are international collaborations."

The engineering panels want academics to be submitted regardless of how long they have been in academia. There have been complaints that earlier RAEs discouraged departments from recruiting from industry or hiring young academics because such staff would have few publications.

Larger panels

In previous exercises, the mathematics and computing assessment panels complained that the output took much longer to inspect in detail than the funding council had anticipated. This time, the panels will be larger.

Nigel Hitchin, chair of Panel F and Savilian professor of geometry at Oxford University, said: "In general, it was felt that larger panels would cope better with the volume and range of submissions and give more confidence to the community that their work would be assessed properly."

Overall, the panels expect to scrutinise at least half the output submissions, but the computer science and informatics panel expects to get such a large volume that it will study only a quarter.

Professor Hitchin insisted there would be coherence. "Individual subject panels are allowed to have different views on certain things, but I don't expect disparity."

Julia Higgins, chairwoman of the physics, chemistry and environmental science Panel E, said that panels had been working more closely than in previous years to ensure comparability. "There has been a lot more cross-discussion at an early stage. My three panels are working very well together, and we have agreed to cross-reference to appropriate sub-panels," she said.

Submissions to Panel E units will be invited to add details of extraordinary impacts of submissions, for example if they have been taken up by industry.

All panels agreed that it would not be a problem for scientists and engineers to enter four pieces of work because the exercise covers a period of more than six years.

caroline.davis@thes.co.uk

Reactions: 'We were forced to play games'

The research assessment exercise will add uncertainty to the future of many science and maths departments, academics have warned, writes Caroline Davis.

The problem with the RAE had been that the funding councils did not confirm how much money would be awarded for different research grades before the results were published, said Robert Curtis, head of pure mathematics at Birmingham University.

"Last time we were forced to play games," he said. "Being awarded a 4 seemed reasonable but when it came to funding, very little money was given to 4s. This made departments vulnerable, and vice-chancellors then had to make decisions."

He said the main desire of the maths community was a clear and transparent exercise devoted to the allocation of research funding, not to league table rankings.

Meanwhile, engineers welcomed the recognition given to applied research in the RAE criteria.

The engineering community has strongly criticised previous RAEs for their emphasis on publication of academic papers. Engineers argue that this does not reflect the applied nature of much of their research.

But Phil Ruffles, chairman of the Royal Academy of Engineering's research working group, said this time panels had taken on board many of the concerns. As well as lobbying the panel for a wider view of research output, the group had called for the RAE to consider the sustainability of departments after reports that many staff in the field were nearing retirement.

"By and large, what we asked for has been included," he said. "We wanted to see a reasonable distribution of age and procedures for developing people's careers. We're happy that with these panels engineering will get a good assessment."

He welcomed the panels' attempts to include industrial outputs but said he would like to see more input from industry in deciding what was included in terms of the impacts of research.

The panel chairs have also tried to ensure that all staff are included in the exercise, irrespective of career breaks or experience.

There was also a welcome for the allowances that will be made for maternity and parental leave and the obligation to take into account health and safety issues, especially with regard to pregnant and breast-feeding women.

Helen Heath, a high energy physicist at Bristol University, was unable to carry out experiments when she was pregnant and her department was unsure whether to enter her for the RAE. She said: "It makes clear that disruption to a career due to the arrival of children should be taken into account. This will encourage departments to include women who might have been excluded from the exercise in the past. I hope the way in which these policies will be implemented will be monitored."

Weightings

Panel E

Earth systems and environmental sciences; chemistry; physics.

• Output: 60 per cent
• Esteem: 20 per cent
• Environment: 20 per cent
• The panel assigned a relatively low emphasis to outputs as it felt context and sustainability were also key.

The panel will request a strategy for six years from 2008 to be considered under environment.

Panel F

Pure mathematics; applied mathematics; statistics and operational research; computer science and informatics.

• Output: 70 per cent
• Esteem: 10 per cent
• Environment: 20 per cent

The panel emphasised outputs saying that this best reflected departmental change and age profile, was simpler to assess objectively and reflected the community's confidence in peer review.

Panel G

Electrical and electronic engineering; general engineering and mineral and mining engineering; chemical engineering; civil engineering; mechanical, aeronautical and manufacturing engineering; metallurgy and materials.

• Outputs: 50 per cent
• Esteem: 30 per cent
• Environment: 20 per cent

The panel felt that because so much engineering research is group based and collaborative, individual outputs should be accorded less emphasis in favour of esteem.

Collaborative outputs would be entered as exemplars or case studies and assessed under esteem.

Successful grant applications and contract capture will be credited under esteem, as will industrial take-up of research.