The difficulty of recording impact accurately (and why it’s so important)

Posted in Blog on 30th January 2020

Whether as a fundraiser, project coordinator or director, if you work for a charity a big part of your role is about being accountable. And this accountability ultimately comes down to one thing: impact. Now, of course different charities will report on different aspects of impact, but fundamentally the challenge remains the same. How do you capture your impact, and why is it important?

What do we mean by impact?

As mentioned before, every charity will have different criteria by which they measure their impact. One aspect we frequently report on is the number of organisations we have worked in partnership with, or the number of care settings (schools, hospitals, day centres) we have reached. But Music as Therapy International’s impact is ultimately all about people. How many caregivers (teachers, community leaders, managers) did we train and support? How many vulnerable people will benefit as caregivers put their training and learning into practice? And arguably more importantly, how has it benefitted them? What changes have we seen?

This brings us to the two differing types of impact: qualitative (outcomes) and quantitative (outputs). The latter deals with facts and statistics. How many people did we reach? How many sets of instruments did we provide? The former, more anecdotal approach refers to all the benefits, positives, changes and impact that are a bit harder to put a number on. It could be a quote from a caregiver or music participant, a photo or video capturing the effect of music on one person in particular, or a case study of how the wellbeing and behaviour of music participants has improved thanks to the introduction of music into their care. 

Neither qualitative or quantitative impact alone can give the full picture, so it’s important for any charity to report on both at the same time. There’s no point celebrating big numbers if you can’t tell the story of the people who make up those numbers. Likewise, one person’s story can be extremely powerful, but if there’s no idea of scale then fundraising, for instance, can be more of a challenge. With that in mind, the question becomes as much about what impact to report on, as how to go about capturing and recording it.

Tracking impact over time

One key aspect of recording impact is also being able to do so over time. We are extremely proud of the fact our projects and activities create a genuine legacy of change. We know, for instance, that some Local Partners have continued to use music with the vulnerable people in their care for more than fifteen years since their initial training. 

However, where it gets difficult is tracking the knock-on benefit of this. If one individual has been using music for over a decade, how can we keep tabs on how many people they work with from year to year, and the difference it has made? It may seem like a fairly simple question with an apparently simple answer, but the truth is while as a charity we are required report to our funders, supporters and the Charity Commission, many of our Local Partners do not need to do this. A such, it may be that at best they only have rough estimates of the number of people they use music with.

Furthermore, as a partner-led organisation, it’s not uncommon for Local Partners to ‘go quiet’ for a year or even more, before surfacing again, possibly even in another care setting entirely. In instances like these are we to include impact figures for the intervening years, or just those during which we were in direct contact?

Historically, we have worked around this problem by focusing our reporting on the impact of our Introductory Training projects in any given year. Yet as we continue to grow and strive to improve our data capture, handling and reporting, it could be argued we should begin to include the accumulated impact of our growing network of Local Partners – a network which will continue to expand in future.

Beginning with 2018’s UK Partner Survey and Sustainability Review Report, we’ve started to introduce a more detailed mechanism via which we are able to collect and report on both our qualitative and quantitative impact. Following its success, in 2020 one of our goals is to replicate the process for our international network of Local Partners. Yet, once again while it may seem a relatively simple undertaking it requires translation, cultural adaptation and may face obstacles such as lower levels of literacy or limited access to the internet depending on which country we are considering.

Accuracy equals integrity

You may ask why capturing our impact accurately is so important. First, in not recording this cumulative impact, there is the danger we are doing ourselves a disservice. As an organisation which places the utmost importance on sustainability, if we are investing supporters’ donations in these activities we should also be reporting on their impact as fully as if they were Introductory Training. Once again, it all comes down to accountability.

Then there’s the fundraising argument (which, as you would imagine is an important one for me). Due to the high level of competition for funding, charities are often chosen on the projected impact of their proposed activities, and it’s here that accuracy is key. If projections are vastly higher and greater than you are able to deliver it may work to your detriment for future funding efforts. Too low or limited, and you might not get the funding in the first place at all. Not only that, but major discrepancies between projected and actual impact can erode trust in your organisation. At a time when both leading charities and the sector as a whole is suffering from record low levels of public trust, integrity is the most powerful tool we have to rebuild that confidence.

Furthermore, we are an organisation with a wide range of stakeholders: care setting managers, caregivers, the music therapists who deliver our training, our volunteers, trustees and donors. Each comes with different perspectives, priorities, aims and goals – what they expect or hope to see from each project. A care setting manager’s priority might be for staff learn new skills and feel more content in their role (our Sustainability Review found 92% of caregivers felt more committed following our training). A caregiver who participates in a project might just want to see the people they care for less agitated and more sociable, while a donor might be keen to see as many vulnerable people reached as possible. Because of this, correct tracking and recording of impact is imperative in order to enable us to effectively and accurately report back to each of our stakeholders with the information they care about most.

In pursuit of excellence

It’s for all of these reasons we are making our impact tracking, recording and reporting a priority for 2020-22 as part of our charity strategy. We will be re-designing the impact recording documents we use to shift our focus from a project-by-project basis to one that looks at partner settings over time. While a small change, we believe it will change the way we think about partner engagement and our reach as well as serving as a more reliable source of data when it comes to reporting. We will also be looking for more opportunities to ask our partners on the scope and impact of their music practice. The international partner survey I mentioned before is one aspect of this. The aim is to ensure our figures are always as up to date as possible as well as make sure we are less likely to miss out on those powerful anecdotal stories and accounts of how music is making a difference in people’s lives. 

We often say that while we remain a small charity, we are also one that is committed to the pursuit of excellence and preparing for future growth. Improving our capture, recording and reporting of impact is just another way we are doing this and, with the help of our Partners in the UK and overseas, growing and working together to make music an integral part of care. 

 

Richard James (Fundraising Manager)

Newer: Advisory Panel Meeting 2020

Older: Our New Strategy for 2020-2022

Share on Facebook Share on Twitter Share on Google+ Share on LinkedIn Share by Email