REF Consultation and Public Engagement

The NCCPE has made an active contribution to the development of the new Research Excellence Framework.

Early development of the REF: 2009 and 2010

You can find details of how we informed the early developments of the REF here.

Development of panel criteria and working methods: 2011

In 2010 the NCCPE was invited by the REF team to contribute a briefing paper to inform the development of draft assessment criteria by the four main panels.  We were asked to focus on two key questions that REF panels needed to address in order to ensure that public engagement was addressed robustly and maintained a credible place within the impact element of REF:

  • In relation to the case studies that submitting institutions will need to submit as evidence of specific impact in the REF: what actually ‘counts’ as genuine public engagement? What is ‘good’ public engagement, what counts as reliable evidence of public engagement that panels should be looking for, and how can panels distinguish between different ‘star level’ ratings for public engagement?
  • In relation to the broader ‘impact statement’ that submitting institutions will need to submit as evidence of the submitting units broader structures, mechanisms and context that facilitate impact what, in general terms, should panels expect to see in relation to public engagement?

To address these questions:

  • We ran a workshop in March 2011, inviting a cross-section of staff from across the sector, with interest and expertise in the assessment of public engagement;
  • We ran a workshop with the Royal Geographical Society to look at the challenge through a disciplinary lens;
  • We were invited to facilitate a workshop with representatives from all the main REF panels in May 2011.

We prepared the following documents to inform these events:

A compilation (PDF 781kb) of seven of the impact pilot case studies, which featured public engagement with research in interesting ways.  These provided the basis for useful conversations about what ‘excellence’ might look like;  

A detailed briefing (1.46mb) about the impact pilot process and the findings from it;

Following this series of events, we prepared this briefing paper (PDF 772kb) for the four Main Panels, addressing how impacts arising from engaging the public with research might be assessed.

Consultation on draft panel criteria and working methods: 2011

The REF team then published, in July 2011, their ‘Assessment framework and guidance on submissions' (REF 02.2011).  This set out the general framework for assessment in the 2014 Research Excellence Framework (REF) and provided guidance to UK higher education institutions about making submissions to the 2014 REF. It included guidance on procedures, the data that will be required, and the criteria and definitions that will apply.  This was not for consultation

At the same time, draft panel criteria and working methods were also published, and consultation responses were invited, with the deadline of 5th October 2011.

The NCCPE prepared another briefing document (1.24mb) which synthesised the published documents and identified the key issues from a public engagement perspective, comparing the different approaches of the four panels.  There was some variation and inconsistency between the panels. 

The NCCPE compiled feedback from our network, and submitted this response to the consultation (PDF 244kb) .

What's next?

The final panel criteria and working methods will be published in early 2012. Taken together with REF 02.2011, which was published in July 2011, they will give a comprehensive description of the information required in submissions to the REF, and how the REF panels will assess submissions.  The deadline for submissions to the REF is 29 November 2013.

The NCCPE will continue to work actively to support the development of the REF.  We are particularly keen to  focus now on the development of exemplar case studies, where we can model effective ways of capturing the outcomes and impacts from engaging the public with research.  We also plan to develop guidance about some of the various challenges, for instance how engagement through the media can be effectively assessed.  If you would like to contribute to this work, please get in touch.  We will also be running a workshop at Engage 2011 on this topic.