A five-layered 'footprint' for assessing engineering research excellence may be invaluable in other sciences, writes Jennifer Gristock
A new approach to research assessment was unveiled this week by a Royal Academy of Engineering investigation into the measurement of research excellence. The process, which can be applied to many different types of scientific research, aims to recognise the wide variety of research outputs that contribute to United Kingdom innovation.
The assessment process has been developed by a working group of academic and industrial engineers chaired by Philip Ruffles, director for engineering and technology at Rolls-Royce. "We started from first principles, asking leading academics and industrialists how they judge quality in practice," said Mr Ruffles. "By qualifying the relationships between engineering, science and technology, we concluded that research excellence should be judged with regard to five characteristics.
"Besides traditional publications, a measure for academic success, the process should also assess research conducted in a context of application, strategy, independent scholarship, and vitality and sustainability. Only when analysing all together will an assessment result in a balanced judgement of the quality of research, the capabilities of the research group and the relevance of the research to its users."
The working group studied research assessment procedures in the UK and overseas. The measures proposed for the 2001 research assessment exercise, those used by the Engineering and Physical Sciences Research Council and overseas assessment methods (see box) were among those used. "Relating these measures to the five characteristics leads to a powerful and versatile tool for measuring the quality of research group capability and output," Mr Ruffles said.
"The assessment process also assists in establishing whether research is likely to have the desired impact on society. As an example, partnerships such as the Link scheme for large firms and the teaching company scheme for small enterprises are indicators of a clear strategy for research within a specific context and demonstrate vitality and sustainability. Using the proposed set of measures, the quality of research is assessed in several different ways to provide a method far more robust than relying solely or largely on publications."
The new methodology yields a footprint showing the five characteristics on an individual axis to generate a visual representation. The shape of the footprint gives the relative strengths in the five characteristics and its area shows the relative strength of the research group and its output. Research groups in cooperative research with small or large firms can be assessed alongside academic groups that focus on an academic audience and secure an equal rating.
"The proposed assessment process has a number of checks and balances that will sustain and encourage the diversity of engineering research in the UK," Mr Ruffles said.
The report, Measuring Excellence in Engineering Research (ISBN 1 871634 86 5), is available free from the Royal Academy of Engineering, tel 020 7222 2688.