Evaluating your project

 

‘Schools and arts organisations need to work collaboratively to provide more and better quality evidence of the outcomes they achieve through their joint arts in education work. This could be achieved by improving their evaluation methods, developing more sophisticated case studies and recording outcomes more effectively. There is also a need to do more to share best practice’.

An independent report for the Welsh Government into Arts in Education in the Schools of Wales, Professor Dai Smith, September 2013

 

How you plan to evaluate your project should be agreed at the outset and included in the project brief. Teachers, artists and learners should all be involved in the evaluation.

Evaluation isn’t something that just happens at the end of a project. On-going monitoring during the project is critical to help refine and improve the project as it progresses whilst the final evaluation should inform future projects and effective ongoing practice.

Depending on the scale and length of your project, your evaluation may be as simple as a discussion between the artist(s) and teacher or a formal detailed report.

Here are some of the things your Evaluation might cover:

  • The project brief and starting context – including the project aims
  • learner outcomes from the project (both planned and unforeseen outcomes)
  • impact on teachers and artists involved
  • any wider impact – eg on the school community / parents
  • an explanation of the approaches and data methods you’ve used to measure impact  - aim for a mix of methodology - quantitative (eg assessment scores) and qualitative (eg feedback from students – verbally or written)
  • Ideally, you should benchmark the participants’ starting points in order to be able to measure the project’s impact

Try to make your evaluation as honest as possible. It’s important to capture positive outcomes as well as identify potential areas for improvement. Evaluation methodology doesn’t need to be just questionnaires and data (useful and valuable though both are). You may find that artists have additional imaginative ways of documenting the project (eg through film, participant diaries etc) that can build up a richer picture of a project’s impact. 

evaluation diagram

Evaluation could be a brief written report (most funders will require this as a minimum) or- in the case of short one-off visits, a discussion / conversation between the artist(s) and teachers.

Building your Evidence-base (impact areas and sources of evidence)

There should be many opportunities throughout the project to collect evidence demonstrating the processes followed and the outcomes in terms of the learners’ progress as well as the success of the project itself.

In relation to the LNF, this might include:

 

Impact: Standards

Sources of Evidence: 

  • Teacher assessments on pupil progress (taking account of learners’ individual starting points and tracking progress throughout and at the end of the project). 
  • Non-standardised tests
  • Standardised tests and marking frameworks (Eg. National Reading and Numeracy Tests, End of Foundation Phase assessments ,End of KS2  results) – however, beware of over-claiming causality between short, one-off projects and end of key stage assessments / markers that bridge 4 years of learner development, during which time learners will have been exposed to a variety of different experiences. Teacher assessments may be a more accurate indicator for short projects.
  • Estyn reports; Internal or external review, (Local authority/ regional consortia reports); Schools Development/Improvement Plans 

 

Impact: Attitudes

Sources of Evidence:

  • Commercial tests and surveys
  • In-house devised likert style surveys – eg smiley charts
  • Drawing by pupils
  • Reflection on photographs taken during the project  

 

Impact: Achievement

Sources of Evidence

  • Portfolios
  • Video
  • Photographs
  • Teacher records
  • School Improvement/Development Plan
  • Pupils books bench-marked and looked at again during evaluation stage;

 

Impact: Skills

Sources of Evidence

  • Conversations
  • School curriculum documents (skills -based approaches)
  • Journals 

 

When measuring impact, you’ll want to focus primarily on evidence and feedback relating to the aims of the project. However, in most projects there will be unforeseen improvements / outcomes in learners’ skills, knowledge and attitudes, which should also be captured.

Evidence can be collected in in a variety of forms including video/audio clips, photographs, digital scans of learners’ work or teachers’ field notes.

Look at the evaluation approaches successfully used in the Zoom and Snow Leopard projects included in the case studies section. You can find further guidance on Evaluation in the Resources Zone/Useful Info.