Where: University of East Anglia
What: Qualitative baseline research
Why: To provide a qualitative baseline against which changes in institutional culture, relating to public engagement, could be assessed. Specifically, the research sought to access views on university public engagement and explore cultural and institution barriers to involvement.
Who: A sample of 55 academic and research staff at UEA.
When: Research carried out between May and October 2008. Results published December 2008
"The starting point for improving engagement in your institution must be understanding your current position."
Community University Engagement East (CUE East) led by the University of East Anglia (UEA) started its work as one of the six Beacons for Public Engagement by carrying out a qualitative baseline research process. This involved a cross-section of academic and research staff at UEA. The inductive research supported the Beacon’s overall emphasis on research and evaluation. The baseline research has been very helpful in embedding public engagement across UEA. It has not only provided a baseline against which improvements can be assessed, but has also recognised the public engagement activity already taking place and identified motivations and barriers for getting involved.
A Qualitative Baseline Research process was one of the first activities that CUE East undertook after successfully becoming a Beacon for Public Engagement. The study aimed to capture a snapshot of academic attitudes towards public engagement and the factors affecting their involvement.
The research process forms part of a wider evaluation of CUE East and provided a good starting measure to evaluate impact of the four year Beacon programme. At the end of the programme the research process will be replicated to see what change has taken place. The baseline study was carried out by an external organisation, the Research Centre, City College Norwich.
It was the first time anyone had asked qualitative research questions about public engagement at UEA, and as such gave UEA staff their first opportunity to express their views. Before the research process this was an un-researched area. The extent of public engagement activity was unknown and the research helped to uncover a lot of “hidden” public engagement.
“This first opportunity to go out there and see what’s happening already in the area of public engagement provided a strong foundation for the CUE East programme”. The evaluator
The research plan initially included both qualitative and quantitative research methods, however, this proved unnecessary as the overall evaluators of the Beacons were already going to undertake a survey. Rather than duplicating the quantitative elements, it was decided to focus resources on qualitative research.
It was important to ensure a diversity of people from within the institution; from those “on the ground” doing this day to day, to others with a more strategic responsibility.Overall, therefore the Beacon used a purposive sampling approach (sampling with particular predefined groups in mind). The study targeted heads of school to get an overview of what was going on in their schools, particularly to get the strategic overview of what level of support was offered. The faculty part of the sample aimed for greater diversity and therefore included a quota approach looking for individuals in the four grade levels (senior academic, academic, senior researcher and researcher).
Sample 1 - targeted individuals (n=24)
Heads of Schools
Associate Deans for Enterprise & Engagement
Associate Deans for Research
Pro Vice Chancellor
Sample 2 - randomly selected academic staff by faculty & grade (n=31)
Four faculties – Science, Social Science, Health, Arts & Humanities
Four grade levels – Senior Academic, Academic, Senior Researcher, Researcher
“Not knowing if the interviewees would have any involvement with public engagement resulted in quite a diversity of views, ranging from people that were very engaged to people that didn’t have any involvement at all and were still keen to participate in the study”. The evaluator
The interviews were semi structured, and explored people’s attitudes towards public engagement, their opinions on barriers to engagement, and levels of recognition, support and reward for public engagement at UEA. The responses were analysed using a thematic approach, structured along a set of key areas. Quotes were pulled out accordingly and coded.
Key research question areas of the qualitative baseline research:
- The meaning of public engagement
- Drivers of involvement
- Barriers to involvement
- Perceived importance
- Levels of involvement
- Institutional recognition, support and reward for public engagement
Participants were also asked a series of questions about CUE East in order to help shape future implementation of the programme.
The results of the study have significantly informed the development of the CUE East programme. In fact, the study produced a very tangible output in the form of a table of the specific gaps and needs identified in the study. This meant CUE East could develop activities that addressed the gaps and needs. The findings of the study also validated the business plan that CUE East had put together at the start of the Beacon programme. The study showed that the activities CUE East had planned were a good fit with the needs identified and the study also helped to develop additional activities. For example, the participants considered the proposals for public engagement awards the most challenging of ideas to put into practice; because the breadth and diversity of public engagement activity might make it difficult to ensure inclusivity. The grassroots feedback that the study provided the planning team really helped shape the structure of the Individual Awards and helped overcome potential pitfalls.
CUE EAST QUALITATIVE SURVEY KEY FINDINGS
CUE EAST AND ALIGNED UEA STRATEGIES
84% involved in ‘self defined’ PE, but lack of a shared understanding.
CUE East devised ‘working’ typology, shaping new promotions criteria at UEA.
PE not as important as research and teaching and for some, admin.
New promotions criteria links PE to teaching & research.
Barriers include time, career progression, peer approval, research-led culture, funding & challenges of engaging people.
CUE East developing infrastructure dedicated to building capacity at all levels.
Lack of strategic support: “Support is personal rather than organisational”
New UEA Corporate Plan with ‘Enterprise & Engagement’ a corporate priority.
Very little recording happening & no institutional mechanism in place
CUE East devised ‘Engagement Tracker’
PE was not rewarded in any formal way.
New promotions criteria and individual awards.
Evaluation of PE was minimal. Most had not even considered evaluating their activities.
Beacon’s Researcher advising individuals & projects on evaluation techniques & methodologies.
Source: Presentation to British Science Association Annual Science Communication Conference, (22nd June 2009)
One of the most exciting outcomes of the research was uncovering the level of hidden public engagement. The researchers found that a lot of people were involved in public engagement in different ways, except they weren’t telling anyone, or didn’t consider it to be public engagement.
“When people started to talk about what could possibly be public engagement, they actually started thinking about activities they were doing and realised it could be classified as public engagement. Some people have been doing these activities for a long time. You are now putting a new label on it, and trying to support it in a particular way. If you would go round now and do this research again, a lot more people would be familiar with the term public engagement then they were in the first round, and that’s what you want to see.” The evaluator
However, the research also found that structural support for public engagement was lacking. There were pockets of support, yet nothing at the organisational level. This helped identify where the CUE East programme could have a strategic impact.
CUE East was one of the first Beacons to employ an evaluator and to carry out baseline research. The evaluation started early. The evaluator was already involved during the early stages of the Beacon application process. This meant the evaluation methods were considered prior to the start of the Beacon programme, which helped ensure that the baseline research could be carried out right at the start of the programme. Other Beacons started undertaking similar baseline research after this. The work of CUE East helped improve the baseline research methodology of some of the other Beacons.
“We had Beacon evaluator meetings, were we could share our work and I found that quite useful. It is good to hear what others are doing and to learn from this.” The evaluator
The Beacons are obviously all different, for example, for CUE East the methodology was quite straightforward because it covered only one institution. Other Beacons encompass several universities; these have to face the challenge of sampling across several institutions. The qualitative research might then be even more selective, and a survey could perhaps be more useful than a smaller sample of interviews.
Successes and pitfalls
- Ensuring that the evaluator worked closely with CUE East to develop the interview schedule and promote the research was useful ensuring that the results of the research could be used in a practical way. Comparing the identified needs with the activities in the CUE East business plan confirmed that what they had set out to do was in fact in line with what was needed.
- Carrying out the baseline research at the start helped inform the development of the programme. Some other Beacons started a similar study when their programmes were already underway. This may have affected the way that the baseline research could be used to influence these programmes.
- The research revealed that there was a hidden general enthusiasm for public engagement amongst staff. The interviews showed that people were interested in finding out how they might get involved.
- A real strength of the research was the sampling approach chosen. The richness of the data collected was dependent on the sample.
- Meeting with the other Beacon evaluators was useful. The work of CUE East helped some of the other Beacons in preparing their baseline research, for example UCL and Beltane. However, being the first of the Beacons to undertake a Baseline Research meant CUE East couldn’t build from previous learning.
- Look at how you could involve other staff in the sample. This research only focused on academics. Even though it would have been useful to reflect a cross-section of all staff, an assumption at the sampling stage about who was involved in public engagement was necessary, in part because of limited time and capacity. The overall Beacons evaluation includes both academic and other staff; this learning could be helpful for future developments.
- Working collaboratively with the institution(s) is hugely important. An external evaluator can add a lot to a research process but they will effectively be an outsider and really needed this collaborative approach to for example gain access to the interviewees.
- Think about timing. This research was done during the summer, which unexpectedly turned out to be the best time in terms of recruiting interviewees.
- Be flexible. It is helpful to work on a flexible timescale (and sometimes use gentle persuasion) to get interviews set up with the academics.
- Sharing the findings with the respondents and following it up.
“Don’t just go and speak to your interviewees and make that the end of it, but keep them informed and included. If they had an interest and wanted to follow it up then you’ve provided the opportunity for that to happen.” The evaluator
In the final year of the Beacon programme the baseline research will be replicated to see how attitudes and views have changed, and what the impact of the programme has been. Do people feel more supported? Do they feel there’s more reward and recognition? Conducting follow-up the research will give a sense of what’s happened, and an indication of the extent of the change.
“You’d hope to see that through the key activities CUE East has been working on, that people are more aware of the agenda and feel more support is available. Some of these structural changes are key to instigating cultural change. To change people’s attitudes you need the structural changes. Hopefully these changes are felt and have an impact on the ground, and people will feel encouraged to get involved.” The evaluator
The evaluator worked collaboratively with UEA and CUE East. For example, with the Human Resources department who provided the staff list, and the CUE East Steering group, who fed back on the results of the study.
- McDaid, L. (2008) A qualitative baseline report on the perceptions of public engagement in University of East Anglia academic staff. Report No. RS7408, The Research Centre, CCN, Norwich Click here to download a copy of the report.
- Qualitative Interview Questions CUE East Qualitative Interview Questions.pdf
- Presentation to British Science Association Annual Science Communication Conference, 22nd June 2009
- CUE East contributed to the National Coordinating Centre for Public Engagement (NCCPE) presentation.
NCCPE Presentation - Moving Forward with PE - solutions BSA 22nd June 09.ppt
Julie Worrall, CUE East Project Director, presentation notes pdf
Name: Julia Stinton
Organisation: The Research Centre, City College Norwich, Norfolk, UK
Telephone: 01603 773215