Has the REF helped to embed support for public engagement?

Impact image

As universities await the conclusions of the REF 2014 assessment exercise, the NCCPE having been considering the impact of  the REF on public engagement, and whether this has been a positive or a negative thing.

In our recent web poll of 181 people, 70 people said that the REF had helped embed strategic support for public engagement in their institution, with a further 48 saying it had partially done this. So on the face of it, it seems that the REF has helped the NCCPE in its mission to embed public engagement into universities and research institutions. However what does this mean?

In May the NCCPE hosted an event to explore this in more detail. What we found was quite encouraging, with delegates commenting on a number of positive outcomes for public engagement including giving public engagement a ‘harder edge’ in terms of its strategic value to the institution, rather than being dispensable, a ‘nice to have’. Delegates had seen an increase of demand from academics wanting to develop more effective engagement, often academics who had not expressed an interest before. Increasingly public engagement is regarded as an essential part of research.

The picture wasn’t all good however. In our poll, 63 people said that the REF has not had an impact on embedding strategic support for engagement with their institution. Also, delegates at our event noted a range of challenges including the impact on the good will of research partners, who were overwhelmed by the demand for evidence of impact; concerns that more instrumental approaches to engagement for REF-able outcomes took academics away from developing more mutually beneficial relationships with their partners, and other important forms of public engagement; and that time invested in auditing could potentially have been more usefully used to innovate. If you want to find out more you can read the full report of the event here.

As more universities invest in people to support research with impact, what is clear is that when the REF results come out, we will have a clearer understanding of how the panels have viewed the impact case studies, and how engagement has fared as a route to impact.

We would be interested in hearing people’s views on the impact of the REF on strategic support for public engagement:

  • How has it changed how your research group approaches engagement?
  • How has the REF affected investment in public engagement activity that does not lead to REF-able outcomes?
  • Who has benefitted from the REF, and how have they benefitted?
  • What have been the costs and benefits to public engagement?

Please share your comments below. We will be picking up this discussion in more depth at Engage 2014 where we are hosting a seminar all about the REF, so look out for that when planning your visit.


As someone who helped a couple of universities to prepare their "impact" evidence, I saw little evidence that they took public engagement seriously, or even understood what it was about. Certainly few researchers included evidence of their public engagement.

I suspect that submissions played played public engagement down because the universities had a much bigger issue to address, the wider "impact" issue. Few researchers understood what REF was asking for when it sought evidence of impact. Even fewer wrote persuasively about their impact.

Let's face it, when a researcher thinks about impact, what comes to mind is the number of papers published, research grants won, conference attended and so on. They simply don't concern themselves with what the wider world, the one that pays to keep researchers off the streets, make of their work. Public engagement is even lower down the list of priorities, except for the tiny handful of researchers who are passionate about their communication to a wider world.

With researchers themselves less than enthusiastic, and with REF itself vague on what it was looking for, is it any wonder that universities failed to put much effort into enabling, or encouraging, their researchers to indulge in public engagement?

I found the REF encouraging in terms of the public engagement activities of my research group, and we submitted an Impact Case Study based on our work. Attending the NCCPE workshop that advised HEFCE about PE for the REF was undoubtedly useful in that regard. But we now wait to hear what the Panel makes of it, of course; my concern is that Panel members might not all be in the same place on the learning curve wrt PE, its value, and how to evaluate it.

Personally, the specific requirements of the REF were not an issue. For example, impacts had to be "underpinned" by specific research outputs, so "general engagement" about a subject area (rather than engagement with our specific research findings) was less desirable in REF terms. But it's easy to use engagement with specific research findings as a "hook" to achieve "general engagement" with a broader subject area. I've also found the distinction between sharing your *own* research with wider audiences, versus undertaking a separate career in "general" science communication (e.g. Brian Cox, Jim Al-Khalili), to be useful in persuading colleagues that valuable PE is possible without becoming a media star.

I also found the concepts of "reach" and "significance" to be useful when evaluating impacts. Overall, I think it has made my group much better at evaluating the outcome of individual engagement activities, by capturing details of our audiences and devising ways to record behavioural responses triggered by our specific interaction. We may all think engagement is worthwhile and enjoy it, but as I now tell PhD students and early career researchers when training them in public engagement: how do you know the audience aren't leaving your talk etc at the end thinking it was crap?

Having invested considerable effort into pulling all the data together for our REF Impact Case Study, it has also been handy to use that evidence elsewhere. For example, we've just submitted it as an entry to the "Impact" Awards competition of our Research Council - we wait to see what they make of it, and whether their perspective of impact is still primarily in economic terms.

I also quite like the REF's pragmatic approach to evaluating the return benefits from the "two-way" nature of engagement. If engagement is good, and returns benefits to us as researchers, then those will be shown in the quality of our research outputs, which the REF already measures separately to "impact". I've previously written about how engagement can make us "better" researchers here (and that perhaps offers a motivation beyond any sense of "duty"), for any interested - http://bit.ly/whyengage

I'm wondering what is meant by engagement in this context? Does engagement, in this context, cover public involvement in research? I wonder if the poll examined whether the REF has helped to embed public involvement in research? Many would describe public involvement in research as being different to public engagement.

The question of how poll users made sense of the term 'public engagement' I can't sadly answer. However from the event it was clear that participants were thinking of it as a relatively broad term covering a range of activities including public involvement in research. That is the NCCPE's view too - we define public engagement as ''the myriad of ways in which the activity and benefits of higher education and research can be shared with the public. Engagement is by definition a two-way process, involving interaction and listening, with the goal of generating mutual benefit.'' Whilst some people specifically use the term public engagement to mean dissemination of research to the public, we recognise that this is only one facet of engaged research, public involvement in research is another, co-production another etc. Our motivation in the poll was to find out how all aspects of public engagement had fared during the REF - but it you want to find out more check out the event report, or join us at the REF workshop at Engage 2014.

Well, all the REF Impact Case Studies are now available to browse via the REF website.

Looking at Biological Sciences, Chemistry, Physics, and Earth/Environmental Sciences, overall just under 10% of submitted Impact Case Studies in those areas involved "public engagement" (if we define that as interactions with media, schools, and groups other than policymakers or businesses).

Incidentally, the proportion of Impact Case Studies involving "public engagement" for Physics (~22.5%) was much higher than for Biological Sciences (~3.9%), Chemistry (~4.8%), or Earth/Environmental Sciences (~8.8%).

Some "beacon" institutions for "public engagement" with STEM don't appear to have submitted any Impact Case Studies based on PE for biol, chem, phys, or Earth/env sci; perhaps because they had stronger non-PE cases (e.g. commercialisation of research) to use.

So there we are: around 10% of Impact Case Studies in those science areas involved "public engagement" as defined above. That's definitely an improvement on the RAE, where PE was largely ignored (unless you could work it into "esteem" somehow), but shows there is perhaps still some way to travel in making PE a routine part of being a research-active scientist. Perhaps if the next REF were to have a separate section assessing PE with research, making some consideration of it mandatory for all institutions in their submissions?

Add new comment

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.