Measuring the impact of Northern Ireland’s charities: a work in progress

22 Jun 2022 Ryan Miller    Last updated: 22 Jun 2022

Best practice has moved from counting outputs to measuring outcomes. However, change takes time. The local third sector is moving in the right direction but more needs to be done.

 

It’s been a tough couple of years for the third sector. An increased demand for services has mostly been met – and met well – in trying circumstances. At the same time, funding and fundraising have both taken a hit.

Charities are stretched to breaking point and the last thing most of them need to be worrying about is spending time or resources on something other than frontline services.

However, every organisation has bureaucracy. Without support structures, it is impossible to function. The better those supports are, the better the organisation will run – and, ultimately, the bigger a difference its frontline services can make.

A lot of back-room detail is vital. One big challenge for the sector is that the number of bureaucratic aspects of good practice can seem to grow and grow. Even if the positive effects of these new concerns are clear, putting the work in can still be a headache.

No, this isn’t about cybersecurity – although that’s just round the corner – but something that’s been in the air a little longer: outcomes.

We are in the second half of a revolution in how we appraise services. The changes being made are fundamental.

Programmes, schemes, initiatives – all of these used to be measured in terms of outputs. How much work are you doing? How many people is it reaching? How many work hours has it taken up? All of these are scales of busyness. However, none of them really measure the most fundamentally important aspect of any service: its results.

Outcomes are more important than output. They are what will decide whether a scheme has been effective or not.

That doesn’t mean outputs have no relevance. They remain a great way to assess efficiency (or, put another way, value for money). A tiny positive outcome is good, but if this was achieved with immense and costly outputs, maybe the work model isn’t that great.

Ultimately, though, modern services succeed or fail based on their outcomes.

This isn’t a completely new approach. Stormont made its own fundamental shift from outputs to outcomes in 2016. The third sector started its adaptation to outcomes before that time. This adaptation is still taking place.

How is it getting on?

Impact Practice

Impact practice is what an organisation does “to plan, understand, communicate and improve the difference it makes in the world.”

In other words, it’s about outcomes.

Last month, the Community Development & Health Network (CDHN) launched a report looking at the current state of local impact practice, with a heavy focus on the third sector.

Impact Practice & Inspiring Impact in Northern Ireland also looked at the effectiveness of Inspiring Impact, a UK-wide programme which ran from 2011 to early 2022 in order to build better impact practice among community and voluntary organisations (note that Inspiring Impact only started work in NI in 2014).

Key findings from the paper include:

  • Impact practice knowledge and understanding has increased over the last 10 years​, however more support is still needed for the VCSE sector in the planning, implementation and evaluation of impact practice​.
  • There are varying levels of understanding of impact practice, even within the same organisations.
  • The key barriers to impact practice are lack of staff, time and resources.
  • The different tools and approaches Government and funders use can be confusing and unhelpful to VCSE organisations.
  • There is a need to promote impact practice as a way of recognising the difference made, the learning from work and its role in strategic planning and not just for meeting funder requirements​.

Conclusions

Per the report: “Impact practice knowledge and understanding has clearly increased over the last 10 years. There is a wider recognition of why it is important and the value of impact practice. This is no doubt attributable to Inspiring Impact and the changing focus from funders and NI Government on outcomes-based approaches.

“Nevertheless, more support is still needed in the planning, implementation and evaluation of impact practice. While this is a particular issue for smaller organisations, there is evidence that there are variances in levels of understanding within large organisations…

“The key barriers to impact practice are lack of staff, time and resources. These are not always considered in projects and funding streams. Budget allocations for impact practice and evaluation are sometimes not reflective of the amount of work required. The capacity may not be available within the organisation or the budget may not be financially appealing for external agencies to complete the work.

“The VCSE sector is facing added pressure due to the pandemic. Some VCSE organisations have closed and others have changed their work significantly since Covid-19. Sometimes there is a focus on impact practice to meet funder requirements rather than its role in strategic planning, recognising the difference organisations make and learning from and evidence-based way of working.

“Government and funders use different tools and approaches to measure impact, which is confusing and unhelpful for organisations in their impact practice work. Some requirements are only outputs or quantitative outcome data which can be difficult to collect and do not always reflect the real difference made.

“Never has there been a more important time to measure impact. The NI Executive is bringing forward an Outcomes-based Programme for Government (PfG) and outcomes-based approaches are being used throughout Government including the new development of the Future Planning Model for Health (Integrated Care System).

“The approaches used and expected contribution of the VCSE sector to measure progress against outcomes is not always clear. The VCSE sector wants to be involved in this work, however, given current financial constraints and the potential end of the Inspiring Impact programme, this will have to be resourced by Government to be delivered effectively.”

What now?

The third sector is off to a start, but things need to improve.

This is not just because this is the way the world works now, and demonstrating good outcomes is important for government and for major funders.

All that is true, but the primary reason for good impact practice is that it works. It makes services better. By measuring what a programme achieves, rather than the actions it involved, we all have a far better idea of what is working, what is not, and how things can be improved.

Retooling an organisation so it can keep track of the impact of its work on a sustainable and ongoing basis takes time and resources, and is not something that feeds immediately into frontline services, but measuring outcomes will ultimately make those frontline services better.

What is clear from this report is that the understanding and implementation of outcomes-based working varies across Northern Ireland’s third sector. It varies from organisation to organisation, and from person to person within any given organisation.

There are ways to make this better. Last week, NICVA announced a new initiative on impact practice, which will feature guidance, training and networking for local organisations.

Focusing on outcomes can make a huge difference to frontline services. Find out more here.

Join the Conversation...

We'd love to know your thoughts on this article.
Join us on Twitter and join the conversation today.

Join Our Newsletter

Get the latest edition of ScopeNI delivered to your inbox.