Two really interesting papers have been published in the last couple of weeks which have really chimed with me, and they have happened to come at a time when my own work is evolving to encompass many of the issues raised, and will perhaps allow me to explore these issues further. Firstly, Counting What Counts, published by Nesta, and written by Anthony Lilley and Professor Paul Moore makes a convincing argument that the arts and cultural sector is neither properly collecting or making good use of data which could help us to understand and increase the social and cultural impact of our work – and importantly could enable us to to unarguably demonstrate the results achieved through the investment of not insignificant amounts of public money.

This I think is really important to all of us working in arts education and the participatory arts sector. Everyone involved in this sort of work has stories about how engaging with the arts has ‘changed lives’. This is sometimes presented in the rhetoric of the evangelist in efforts to get across just how important our work is – only last week I attended a presentation in which colleagues who deliver an inspirational arts education programme in the States talked of ‘rescuing people’ through music. I don’t doubt any of these stories, or the essence of the many evaluation reports I see which assert that arts programmes have had effects on people’s lives which extend way beyond developing their artistic skills. We use reports and evidence like this to make the case for funding and investment in our work, and we are often required to produce them by funders to show the return we are delivering on their investment. And of course it is vital that we do hold ourselves to account, when we are usually spending public funds or other people’s money. I may write another time about the way we do this, the sophisticated models that we do have in place, and we do have many – from extensive quantitative data collection and analysis, to those individual and often inspiring case studies.

However, amongst the issues raised by the Counting what Counts is the fact that this so often happens in isolation. We don’t have mechanisms or infrastructure in place to look at the really big picture – the ‘Big Data’ which has the potential to really change practice and influence at the highest level. The other primary issue is related to what we do collect and why, and the fact that it is, as mentioned above, so often seen as the way of justifying our work and existence, and not as a fundamental tool which we can use to inform and shape decision making.

In recent months a combination of things, some work related, have led to me paying more attention to science stories in the news. Another thought I had, whilst reading this paper was how many arts project evaluations are actually subjected to the rigorous standards that any scientific study would be before being published and accepted by the scientific community. I am sure it happens and that there are pockets of good practice, but there is a strong case to be made for an arts education resource and training provision which helps organisations with areas such as statistical analysis. How often has it been demonstrated that the effectiveness of an arts project in achieving social impact is statistically significant? – I’d be really interested in any examples of such evaluations. There is also a case to be made for a peer review system as is standard for publication of all scientific papers.

With the Counting What Counts report still very much on my mind, I was really excited to read that Dr Ben Goldacre had written a paper advocating for use of random controlled trials in educational contexts, and, again for the building of an infrastructure echoing the now well established mechanisms that exist in the medical world. The paper speaks for itself and can be read here. If the arts education world were to take some of these points on board we could really add weight to those inspirational case studies and be in a position to make scientifically proven arguments about the effectiveness of our work in schools, communities and other contexts in addressing social issues and genuinely making the world a better place.

This as ever will take time, but I’d urge anyone who has the opportunity to do research and evaluation around their work with the arts to think as scientifically as they can, and I hope that the Arts Council and other major bodies take both these pieces of work seriously and are able to help the sector build the infrastructure and develop the skills which will be required for the realisation of the potential impact of these pieces of work.