NCCPE publishes draft response to the REF consultation

Consultation on the second REF

The NCCPE today published its draft response to the REF consultation.  Closing on the 17th March, this consultation sets out a variety of recommendations for REF 2021.  Building upon last summer’s Stern review of the REF the consultation also asks for suggestions about how public engagement might be framed in the next REF and how the guidance might be improved.

We will be discussing our draft response at an event on the 7th February in London, where we will be joined for part of the day by Steven Hill, Head of Research Policy at HEFCE.  A few places are still available.

The NCCPE response is supportive of the overall thrust of the consultation, which makes the case for maintaining continuity with REF 2014.  Our response provides suggestions for how various elements of the REF might be enhanced, including:

  • The guidance about public engagement (building on our review of the role of public engagement in REF 2014)
  • The guidance about enhancing collaboration between HEIs and external organisations
  • The suggestion that the definition of what constitutes ‘underpinning research’ in an impact case study be broadened beyond conventional research outputs

While we agree with the intention to maintain continuity of approach with REF 2014, we argue for significant changes in three areas: 

  1. Increasing the involvement of external ‘stakeholders’ in the process of developing and delivering REF 2021.  The REF is currently framed as a process managed by and for the HE sector, with some limited involvement of ‘research users’.  We argue for a more ambitious framing of the next REF, as a more open conversation with wider society about the value of research.  We identify two areas where a more open approach should be explored:
    • Opening up the development of impact criteria to greater external input, through a more active approach to consultation and engagement

    • Reviewing the process of recruitment of panel members, to address ambiguity in the roles of ‘research users’ as representative or otherwise of their communities

  1. Taking a more systematic approach to clarifying the alignment between REF, Research Council investments in research impact, other innovation policy and the TEF.  There remains significant ambiguity and complexity in the relationship between REF and other sector funding policies, which risks creating duplication and inefficiency.  To address this, we argue that the assessment of ‘wider impact’ in the REF should be re-framed as the assessment of excellent knowledge exchange (building on HEFCE’s definition of knowledge exchange as ‘the multiple interactions between HEIs and businesses, public services, charities and communities to create societal and economic benefit’.Our rationale for this is as follows:
    • Identifying that it is the processes of research and KE that are being assessed (using indicators of quality and impact appropriate to both) makes it possible to construct a more coherent rationale for the REF, and better articulate its relationship to other policies and funding regimes.  

    • Foregrounding knowledge exchange as the process and paradigm within which ‘wider impact’ is being assessed,  helps articulate a ‘broader and deeper’ account of what impact is and why it matters, and thereby to address many of the concerns about the overly linear and mechanistic model embodied in the current guidance.

  2. Drawing more purposefully on the extensive literature about engagement and impact to develop a more robust and theoretically informed approach to impact in REF 2021.  There is a wealth of theoretical and practical literature exploring the challenges of impact assessment and evaluation, both within the HE sector and beyond. One recent example is the AHRC report ‘Understanding the Value of Arts and Culture’ which contains a monumental review of state of the art approaches to assessing cultural value.  We would like to see:
    • A concerted effort to increase the sophistication of our collective understanding of knowledge exchange, engagement and impact

    • Recognition that the sector’s ‘impact literacy’ is currently limited, and that concerted efforts will be needed to stimulate robust debate and encourage an intelligent, reflective and critical approach to the development of impact policy, so that the process doesn’t stagnate into a ‘box ticking’ exercise

Please do circulate our draft response to colleagues and feed back your reactions by 10th March, to inform our final response.  You can contact us at