Produced by People Science & Policy Ltd for the Research Councils UK, BIS and the NCCPE (2011), this evaluation guide is aimed at anyone who wants to engage with the public. The first section provides an overview of evaluation - what it's about, what it can do, what it can't do, how you should report your findings and how it can help strengthen funding applications - and looks at building an evaluation strategy. A range of evaluation methods are examined in the second section, with guidance on data analysis and writing evaluation reports.
Published in 1996 by COPUS, this guide for organisers evaluating public understanding of science events details the pros and cons of various evaluation tools. Sections include: The Why?, What? and How? of evaluation; Designing questionnaires; Interviews; Focus groups and Analysis & reporting of results. Ben Johnson's companion Good Practice paper Effective Evaluationprovides a brief guide to some of the most commonly encountered issues in evaluation, and some of the ways in which they are resolved.
In this chapter from the Wellcome Trust publication "Engaging Science: Thoughts, deeds, analysis and action", engagement experts Ben Gammon and Alex Burch stress that effective evaluation, that which will inform and improve practice, depends on clarity of purpose and commitment from the organisation. They place value on evaluation that measures immediate impact rather than on wasting time and money trying to assess long term impact.
The Royal Academy of Engineering's Ingenious scheme is a funding programme to promote public engagement in engineering. Although intended for Ingenious grant holders, this toolkit contains a brief, accessible evaluations guide, which breaks evaluation down into seven simple steps. There is a useful overview of evaluation theory, methods and design; example questionnaires and interview schedules; links to other resources and the Ingenious 'Final Project Report Form'. This evolving toolkit invites grant holders to update with case studies and good practice examples.
Produced for DIUS and the ESRC, this revised version of the Tavistock Institute's Evaluation Framework is not a comprehensive manual on how to conduct evaluations (see the other technical resources listed in this section for this). Rather, it provides an overview of evaluation for Science and Society initiatives along with a set of procedures, guidelines, methods and examples to improve evaluation design. Its primary relevance is to those designing and implementing evaluations of Science and Society activities, from 'public awareness-raising' actions to educational initiatives aimed at encouraging young people to study science and related subjects in school. It is also aimed at policy-makers and programme managers to help them specify the kinds of evaluation approaches that need to be built into programmes.
Evaluation consultant Ben Gammon stresses that clarity is the key to effective evaluation, both in terms of planning and reporting. The pros and cons of different evaluation methods for museum exhibits - observation, interview/questionnaire and automatically logged data - are presented, along with the key features of a well-written evaluation report.
Evaluating community engagement and participation in decision making
The Evaluation section of the URP Toolbox provides a brief overview of the aims and methods of community engagement evaluation, together with a list of the sorts of questions you might want the evaluation to answer.
This guide is intended to help those involved in running or commissioning engagement activities to understand the different factors involved in creating effective public participation. It helps planners set and measure attainable objectives, evaluate impact, and identify lessons for future practice. Using clear language, simple instructions, illustrative case studies and a glossary, this guide is a tool for anyone involved in public participation in central government and beyond.
Produced for the Joseph Rowntree Foundation (2000), this audit toolbox looks at ways of assessing levels of community involvement in area regeneration initiatives. There are tools and appraisal exercises for measuring: the history and patterns of participation; the quality of participation strategies; the capacity within partner organisations to support community participation; the impacts of participation and its outcomes, and the capacity within communities to participate.
The Evaluation Trust is a community development agency that works to disseminate skills and good practice in participatory evaluation to voluntary and community organisations. This Evaluation Toolkit contains advice on different methods and how to apply them; resources, such as visual aids, and advice on choosing an evaluation consultant. It is worth browsing the whole site for general information on What is Evaluation and definitions of common terms.
Evaluating Arts Projects
The Arts Council's (2004, revised) Partnerships for Learning aims to help those involved in artseducation projects to understand evaluation clearly and to evaluate effectively. In the long term, it aims to raise the standard of arts education projects. This guide provides a flexible framework, applicable to many different situations and useful for evaluating short or more extended projects. It will benefit those experienced in evaluation techniques as well as beginners, who may find the checklists and case studies useful. An Introduction to Self Evaluation accompanies this document with a useful list of further resources.
This Arts Council England (2006) guidance document provides a self-assessment framework for those running arts activities with children and young people. The defining characteristics of high quality arts opportunities are described with examples, and prompt questions to help you reflect on your own experiences. The document helps to define good practice and provide advice on developing and delivering arts experiences. The guidance could also be used for project planning and evaluation; analysing support, training and professional development needs, and helping to select artists for participatory work. Section 8 deals specifically with monitoring and evaluation.
Written in 2004, this toolkit aims to help voluntary and community arts organisations in Northern Ireland to carry out their own evaluations, especially the social impact on participants. Many arts organisations in Northern Ireland already have excellent evaluation systems and this toolkit aims to increase the consistency of evaluation work so that individual arts organisations can better understand and explain the impact of their work. The toolkit is the first stage in a larger process that will a) see evaluation extended to all sectors of the arts, and b) will evaluate the impact of arts organisations' work on audiences as well as participants.
Evaluating Work with Children & Young People
Participation Works have published a set of three evaluation guides that focus on work with children and young people. Evaluation in a Nutshell is a short introductory guide that breaks down the evaluation process into simple steps. More detail is given in The Guide. The Toolkit presents a compilation of sample forms and activities for use with children of all ages that you can adapt or copy to collect information during your evaluation. In addition to consent, feedback and monitoring forms, The Toolkit includes the National Youth Agency's Hear by Right Self Assessment Tool, to map and plan young people's participation, and the What's Changed Participation Outcomes Tool to monitor impact.
Published by UNICEF (2005), this comprehensive toolkit includes: classical evaluation tools; tools for participatory research and for participant evaluations of workshops; ice-breakers and team-builders; and a note on body language. Each tool is presented in a user-friendly format to enable the practitioner to put it to use, with information given on: its purpose; instructions on how to use it; the time required; materials necessary; sample questions that it might be used to answer; its advantages and disadvantages; tips, and a real-life example. The toolkit also includes several tools for participants to evaluate the workshops in which they are participating.
Published by the Carnegie Young People Initiative, a research project that aims to improve the quality and increase the breadth of young people's participation in public decision-making, this report provides an overview of the existing evidence about what works in youth participation within the public realm. It examines both the impacts of involving young people and the processes involved. The cited evaluations and research illustrate common themes in existing practice, and where gaps are identified, recommendations for the planning of future research are examined.
This toolkit produced by the Open University for the Blast! Science engagement programme offers a brief outline and introduces some ideas for conducting an evaluation using a fictional project designed to engage young children and their grandparents with chemistry.
See also the Arts Council Guidance document Providing the Best included in the Evaluating Arts Projects section as this is specifically aimed at those organisations and individuals who provide arts experiences and activities with children and young people. The checkliston page 24 outlines to policies and procedures that need to be in place when working with children and young people.
Evaluating Social Impact
Produced by nef (the New Economics Foundation) for organisations with a social enterprise mission, this site will get you started on evaluating your work through a series of simple to use tools. The interactive tools and How To guides will help you measure impact and demonstrate the quality of what you do. As you create a personal toolkit from the list of choices available you can consult The Advisor whenever further advice is needed. Additional practical guidelines, tools and case studies can be downloaded from the comprehensive, regularly updated Library.
Written for recipients of Heritage Lottery Fund grants, this in-depth guidance paper (2008) will help you develop your plans for evaluation. Before you start, you may wish to read First Steps in Evaluation (2009), a brief introduction to evaluation for HLF grant applicants. Both pieces of guidance have been written in conjunction with nef (the new economics foundation) who have wide experience in working with community groups and funders in developing evaluation methods.
The W.K. Kellogg Foundation (WKKF) works to improve the lives of children and families in the Americas and Southern Africa through projects that focus on Education & Learning; Food, Health & Well-being, and Family Economic Security. Written primarily for WKKF grantees, the Evaluation Toolkit offers useful guidance for anyone seeking to design an effective, useful evaluation. Based on WKKF's mission and evaluation philosophy, the toolkit has seven sections: Where to start; Evaluation approaches; Evaluation questions; Evaluation plan; Budgeting; Hiring & managing evaluators, and Additional resources.
The Museums, Libraries and Archives Council's Inspiring Learning Framework aims to help itsmembers capture and evidence their impact by identifying generic learning and social outcomes for individuals and communities. Be sure to check out the invaluable download section with action planning templates and checklists, guides to selecting research methods, questionnaires and coding tools as well as the searchable directory of case studies. Guidance is also given on how the information generated from evaluation can be used to develop new partnerships and support a funding bid.