Mid-term evaluation of FP6 instruments praises ambition, but sees room for improvement on implementation

Brussels, 05 Jul 2004

Ramon Marimon, Spanish Secretary of State between 2000 and 2002 and chair of a high level expert panel charged with carrying out a mid-term evaluation of the new instruments introduced for the Sixth Framework Programme (FP6), presented the panel's findings to the informal Competitiveness Council on 3 July. The report contains a number of recommendations on the Integrated Projects and Networks of Excellence, some of which can be implemented in FP6, and some of which should be introduced in FP7, Dr Marimon told CORDIS News.

As highlighted in the report, the scientific community had high expectations of FP6, and in particular the new instruments. As these expectations have been only partially fulfilled, sections of the report appear rather negative. However, as emphasised by Dr Marimon, participants in the study were generally very positive about the new instruments in terms of their objectives and ambition. 'The criticisms focus more on implementation and perceptions regarding size.'

'The most important insight to come out of the study is that we must get more in line with what the scientific community is willing and able to do,' said Dr Marimon.

The 12 recommendations drafted by Dr Marimon's panel cover a wide range of issues, including clarification of the instruments' goals, increased flexibility for participants, critical mass, the difference between Integrated Projects and STREPs (specific targeted research projects), support for innovative research groups, small and medium sized enterprises (SMEs) and project evaluation.

Two of the principal concerns of those who participated in the study are the costs involved in submitting a proposal, and perceptions regarding how large a project consortium need be.

'It is a common misconception that the New Instruments should be very large,' states the report's executive summary. ''Critical mass' depends on the topic, the thematic area, the participants and the potential impact and added value. The concept of 'one size fits all' should not be applied across all thematic areas and Instruments.'

This was a point echoed by Achilleas Mitsos, Director-General of the Commission's Research DG, when he spoke to CORDIS News on 1 July: 'The distinctive characteristic of each instrument is not its size. It is its scope,' said Dr Mitsos. 'Networks of Excellence, Integrated Projects and STREPs have different objectives. They have different raisons d'être. So it is the scope that leads to the definition of the instrument and not the size. We don't want big things [...]. To give you an example of what I am trying to say, we must be able to have a STREP, for example, that is bigger in size than an Integrated Project.'

Referring specifically to the Networks of Excellence, Dr Mitsos added that the Commission's 'good intentions to have something simple' - basing funding calculations on the number of researchers involved and the type of research - 'has led people, along with this insistence on size and big things, to think that they must come with huge proposals with too many researchers and laboratories and much less integration. You can integrate two or three or five, but you can much less integrate 50 or 70 laboratories.'

Dr Mitsos emphasised, however, that the Commission has already improved things in this regard. 'The second call was much clearer and the opposition much less,' said Dr Mitsos. 'It's mainly our fault,' he added. 'When you write something and the reader reads it differently, it's never the fault of the person who is reading it.'

The mid-term evaluation report also raises the concern that prescribing certain instruments to certain calls is encouraging some consortia to adapt their proposals so that they have what they perceive to be a higher chance of receiving funding. 'People distort what they want to do in order to get approval,' Dr Marimon told CORDIS News. He therefore recommends that researchers themselves select the instrument that is most appropriate for their research goals.

'The European Commission should specify the portfolio of Instruments available and the strategic objectives. Participants should define the specific research objective they will pursue and why this can best be met by the Instrument they have chosen,' states Recommendation 3 in the report. 'We should give more voice and weight to the participants,' added Dr Marimon.

The panel found the costs and risks of participation in FP6 to be 'unreasonably high'. For this reason, the members propose implementing a two-step evaluation procedure. This would involve potential participants first submitting a short proposal, which would be evaluated according to a limited set of criteria including adequacy and excellence. 'Once they know they have a chance of being funded, then they can put things in detail and invest more in the proposal,' Dr Marimon told CORDIS News.

For now, the work of Dr Marimon and his fellow panel members is done. Their mandate of evaluating the effectiveness of the new instruments, as requested by the European Parliament and the Council during the negotiations on FP6, has been completed. 'I think our honest work has been welcomed,' said Dr Marimon. He is pleased with the result, which he says is a report based on more than just opinions. 'We really got the findings,' he said.

He now awaits an official response to his work from the Commission, the content of which will dictate whether or not the panel needs to draft a reply. On the Commission's part, 'follow-up does require a change in procedures,' concluded Dr Marimon. Judging by the comments of Dr Mitsos, who used the phrase 'continuation without dogma' to refer to the Commission's intention to address the report's criticisms and make changes where appropriate, procedures are already being reassessed.

For further information on the review, please visit:
http://www.cordis.lu/fp6/instruments_rev iew

CORDIS RTD-NEWS / © European Communities
Item source: http://dbs.cordis.lu/cgi-bin/srchidadb?C ALLER=NHP_EN_NEWS&ACTION=D&SESSION=&RCN= EN_RCN_ID:22282