Monitoring and Evaluation

This information sheet looks at effective ways to monitor your work, evaluate what you do, and share results with your organisation's partners, funders and staff. It is increasingly becoming a standard requirement of most funding bodies to demonstrate how well organisations are doing, and what's working for them. It is often seen as time-consuming, and unrewarding,; but done well, it can be another creative and enjoyable aspect of your work, and an invaluable system for improving your services. By following this guide, you will be able to treat it as an integral part of what you do, using resources and skills you already have.

What do the terms mean?

Monitoring

• is information-gathering that takes place throughout the life of the project.

Monitoring will help you answer questions about your project, and will be crucial when you want to evaluate. It is important you collect the information in a well-planned and organised way.

Evaluation

• is the assessment of the project, usually once it's complete.

Evaluation uses the monitoring information you have gathered to make judgements about how well you are doing, i.e. what is and isn't working. It will reveal how well funds have been used and the benefits to all those involved. It will also help you create more successful, well-managed projects in the future. For the best results this process should be open and honest.

Self-evaluation

• is when an organisation uses its own people, skills and resources to carry out evaluation of their own work

This ought to be built into the everyday activities of a project so that it becomes a standard part of what you do. All those involved should be aware of what you are trying monitor and what the targets are.

Project
Throughout this briefing 'project' is used as a general term for the activity or event that you are evaluating. However, please bear in mind that these same principles can be applied to all sorts of aspects of your work.

Why do you need to monitor & evaluate?

• To review what you've done
• To measure progress and identify failures
• To make improvements to your work
• To show the impact on your community
• To show volunteers and staff the value of their work
• To develop the group's strengths
• To keep control of your finances
• To show your funders that their money has been spent in the right way. Many will ask you to monitor your work to provide them with 'proof' of success.

Types of Evaluation
There are two types of evaluation you will find useful:

• Process evaluation will assist you in improving effectiveness before and during the event. It involves constantly checking that all is going to plan and questioning whether anything can be done more effectively.
• Retrospective evaluation is a way to measure the event's actual performance. After the event good questions to ask are: did everything go as planned? were your objectives achieved? were your team, participants and audience happy and satisfied? what could you have done differently? and what lessons did you learn for next time?

The M&E Glossary
Some of the terms used in M&E, which you might come across when communicating with funders aren't always familiar. A few of the most common are listed below:

• Baseline: What exists at the start of a project. This can help show gaps in provision or prove the need for the project to happen.
• Demographics: Details such as age, sex, ethnicity and place of birth.
• Inputs: Time, money and resources that go into the running of the project.
• Outcomes: Changes that the project has made to the group or community. These can include increased skills, confidence and knowledge amongst the people in the group.
• Outputs: Measurable evidence about what happened as a result of the project. They show how close to your original targets you were. For example, if you set out to do workshops with 20 young people, what numbers did you actually get?
• Quantitative or 'hard' data: This is the numbers bit - i.e. figures that show what happened during a project. You can find out how many people participated, of what age, where from, what they liked/disliked, what they spent money on etc. Statistics like these help you do an audience profile that can show up gaps in provision as well as new opportunities. This "measurable" data is collected through ticket sales, tick box forms, clickers and other methods.
• Qualitative or 'soft' data: This shows people's experience of the project, including the audience, participants, artists and organisers. Although it isn't "measurable" data, it can reveal the impact a project has on people's lives or on the local community. Did people learn new skills, gain confidence or make new friends? Has the project made the community feel closer? Information can be gathered through photos, video and talking to people in an informal way.
• Performance Indicators: these are highlights that happen throughout a project, indicating points in time or achievements. Having a number throughout the life of a project helps you measure that you are on course for achieving your goal.
• Target Group: the main group or groups you are working with and the people your activity, service or event is for. Your project will probably attract different kinds of people but it is important to be clear about the people who are most likely to participate in your activity. Which groups will benefit or change as a result of the project?

The timescale
During any project, time is one of your most valuable resources, and one that fast slips away the further into the action you get. The key to successful evaluation is to start the monitoring process even before the project begins. Perhaps the easiest way to divide up your time is to plan in three stages - before, during and after your project.

Before
This is the time to assess what you already have and what you will need before the project begins. This means that you can establish a baseline* level from which the project is starting, allowing you to see how far you have travelled by the end. Asking the following four questions will help you do this:

1: What is the demand?

2: What are your aims and objectives?

3: Your resources - what do you have and what will you need?

4: What are your performance indicators?

The next section will look at the different information you can gather about your indicators, and the tools to measure and record it.

During
During the course of the project, you will need to carefully monitor your progress, looking at who is taking part and how many people are attending, who's contributing what, and what they have gained from it. This information will help you evaluate the success of the project at the end.

1: Who's involved? - Measuring attendance

2: How good do people feel? - Measuring responses at the time

3: Who's contributing what?
4: Feedback time - Measuring responses afterwards

After:

The Evaluation
Once you have collated all the information you gathered during the project, you should begin to make comparisons with your estimates from before it began. This is when monitoring becomes evaluation - the time for a detailed examination of the material you've gathered, and for reflecting on what it's telling you. To help you with this process, consider the following:

1: Did you meet your aims and objectives?

2: How much did it all cost?

3: Time for changes?

4: Time to share out the results!

No! Don't put it away for a rainy day!
The most important thing about M&E is that once the process is complete, you learn from your findings, and put them into practice. Remember that evaluation is meant to be used. The results must be seen to be acted upon - otherwise you will have wasted all your participants' time, as well as your own - and the process becomes meaningless. Agree how you will feed back what you have learnt into your daily work and the project's future development.

North East England Monitoring and Evaluation toolkit

A monitoring proforma has been developed by ONE North East and its partners, to assess the economic impact of festivals and events across the North East.

A number of areas will be recorded: from event attendance, to jobs and learning opportunities created, and marketing / PR value of the coverage gained by the event. The collation of this data shall provide evidence on how festivals and events across the North East potentially support the economy of the region.

Currently this is only compulsory for Culture10 festivals and events to complete but it is useful framework for other events to consider as what monitoring and evaluation information to collate.

This framework can be found in the strategy section of this festivals and events toolkit web site. SQW Monitoring and Evaluation

Further Information

More basis advice - "talk to your funders and partners about what they want tyou to measure - then make sure you do it! After all if you fulfill their objectives you may be able to go back to them for more funding next year ...."

For other sources of help on Monitoring and Evaluation, here are some useful websites:
- www.ces-vol.org.uk - The Charities Evaluation Service. They have produced a useful booklet, 'First Steps in Monitoring and Evaluation' which you can download for free from the site.
- www.evaluationtrust.org - The Evaluation Trust