Making impact measurement work for funders and providers of youth services

Kevin Franks is the Programmes Director at Youth Focus: North East and is the lead for the Centre's North East regional network. Kevin has 25 years’ experience in statutory and voluntary sector youth and community work, which includes centre based, outreach, detached and schools work. This blog is adapted from Kevin's talk at 'Funding Change: Making impact measurement work for funders and providers of youth services', held on 21 March 2018 at the Leeds Rhinos Stadium.

The idea of impact assessment within the youth sector is not new and both funders and youth providers have come a long way in measuring impact over recent years. However, that doesn’t mean it is always done well, correctly or meaningfully. Comments and questions, we often hear, range from:

• Are we just collecting data for funders?
• Are youth organisations assessing impact of their own accord to improve provision, or only do so because it is a requirement of funders?
• What happens to all the data that is we collected?
• It is a fuzzy inaccurate discipline.
• We don’t have the resources or expertise to meet funder and commissioner impact. measurement expectations.

And we commonly hear the term ‘tick box exercise’.
When I recently posed the question of “what are the challenges around impact assessment and evaluation between funders, commissioners and youth organisations” to a range of colleagues from across the sector in the North East; responses fell into two main areas:

• We “Just don’t know what funders want”.
• There is “Inconsistency in what information funders require”.

We know and understand there is an intense pressure on commissioners, funders and youth organisations to deliver ‘improved value for money’ and ‘better outcomes’. Interventions cost money and we need to know whether, or not, that money is being spent to produce the best outcomes for young people. This raises the quest of what we mean by ‘value for money’?

In a recent debate on volunteering in the House of Lords the issue of the National Citizen Service representing value for the taxpayer was raised. This was in relation to senior staff salaries and surplus of income over expenditure against participation targets for the programme, potentially being missed by as much as 40 per cent. However, we do know that young people participating in the National Citizen Service Scheme can have a quality experience and many of them report a high level of satisfaction. Focusing on purely the cost of something, as being the best value, runs the risk that there is always going to be someone, somewhere, who’s prepared to do it for less. And if you take price as your main distinction, you’re in a race to the bottom, in regards to quality.

There is value in youth work, and I am sure we all have examples of where being involved with youth work has transformed young people’s lives for the better. However, it is a challenge that this value is easily captured in terms of cost-benefit ratios.

Quality is subjective, whereas quantity is not. Quality can be disputed, questioned and challenged. One cannot dispute quantity – it is easily measured.

Focusing on the price makes it easy to miss the real value – and can turn complex decisions based on ethics, culture, empathy, and understanding of society into much simpler games based on numbers and calculations.
Given the challenges, how can these be overcome going forward. Personally, I believe there is 3 main areas for consideration:

1) Be Clear About Youth Work
Youth work is a distinctive field of practice that puts young people at the centre of the work, and starts from their concerns, their interests and their own starting points. Young people engage in youth work by choice.
The great strength of youth work (and youth workers) is its capacity to adapt, change and grow. However, how many funders, commissioners and indeed the general public really understand what youth work is and what it achieves? How much consensus is there in the youth work sector on the ‘purpose’ of youth work?
We need to be clear about the outcomes we claim youth work can achieve:

• Is it reasonable for youth work to claim it can prevent teenage pregnancy (or even conclusively conclude that this would be a positive outcome)?
• Is it reasonable to claim it is responsible for a young person gaining employment?
• Is it reasonable to claim it is responsible for a young person being ‘less’ at risk of being sexual exploited?

If we do believe that youth work is responsible, or at least contributes, to these outcomes, and others, then surely we have a responsibility to provide evidence to back our claims.
​ 
If we expect young people to invest their own time and effort participating in youth provision, it is only reasonable that we make an effort to make it worth their while. And surely part of this is investing some of our time and effort in evaluating if we are providing a quality service.

2) Shared Language There is an obvious need to support the development, agreement and acceptance of a common language and framework to describe what the youth sector does. This will better enable commissioner and funders to understand what youth work is and support them to invest in quality provision that will provide the best outcomes for young people and communities. A shared common language will also allow delivery organisations, including the local small voluntary ones, to clearly communicate where they sit within the diversity of the wider youth sector and ultimately enable them to be better able to articulate their value.

3) Collaboration
There are a number of ways funders, commissioners and youth organisations can engage with each other.
Commissioners can see the value of the youth sector as a critical player in developing ‘asset-based’ approaches to providing high quality support, and by engaging youth organisations as partners in co-production of outcomes.

Evidence gathered by commissioners and funders can be better shared across the youth sector. Good evidence can be used to confirm or challenge approaches and interventions and to examine which features make them successful and worth investing in.

More can be done to build relationships between commissioners, funders and youth organisations. For instance, invite your funder to come and visit your organisation and see for themselves the work at first hand and hold events that create space for open and critical dialogue between all parties. 

However, these approaches will require a level of courage from all involved. They will require strong and mature relationships, both within the sector, and between the sector and commissioners. These relationships will require time and attention to develop and maintain.

The youth sector has a role in coming together to provide a strong and unified voice. This requires leadership from within the sector to manage competition between different organisations.

We know the youth sector is diverse in its interests and organisational forms and, at times, struggle to (or refuse to) speak with one voice.  Yes, difference of opinion is good. And we should always be open to critiques and different perspectives. However, we if can’t agree on some fundamental issues then we will be forever doomed to remain in this static state of being seen as a second class provision for young people by the state and general public. If we want consistency from commissioners and funders we have to consistency from our own sector. And surely the best way to do this is to have a strong, united youth work sector, delivering quality interventions that enable young people to succeed and thrive.

Quality youth work is a process of continuous evaluation and learning – both for young people and practitioners.

Quality youth work equals quality outcomes for young people, communities and society as a whole.
​
Our young people have a right to the best quality interventions - and we have a duty to provide them.