Youth work beyond the measurement imperative? Reflections on the Youth Investment Fund Learning Project from a critical friend

In this blog Tania de St Croix, Lecturer in the Sociology of Youth and Childhood at King's College London, offers her thoughts on the Youth Investment Fund Learning Project, which the Centre is leading with NPC and others. You can find out more information on the YIF Learning Project at https://yiflearning.org.
​

Many involved in the youth work field are critical of the youth impact agenda, particularly its emphasis on the quantitative measurement of outcomes for individuals, and its neglect of process, group work, and structural inequalities. Those of us involved in ‘In Defence of Youth Work’ have argued that the contemporary emphasis on impact and outcomes cannot be separated from its context, the neoliberal ‘desire to financialise human existence’, and its consequences for which practices are valued and who gets to decide. We have claimed that open access youth work is particularly unsuited to outcomes based management, and that open youth work's future existence is undermined by an emphasis on impact measurement.
 
While those of us making a political critique of impact measurement (within and beyond young people’s services) face an uphill struggle against dominant understandings of ‘what works’ and ‘what counts’, there has been a growing recognition of the specific challenges in evaluating open access youth work. In this context, it has been interesting to follow the development of the Youth Investment Fund (YIF), a £40 million government (DCMS) and Big Lottery Fund investment in open access youth work. While we might start by noting that £40 million over 3 years is dwarfed by a decade of youth work cuts, the YIF is nevertheless significant: it suggests that someone, somewhere in policy recognises the potential value of open youth work. The YIF is also significant in relation to impact debates, as it included “an explicit objective to strengthen the evidence base on the impact of non-formal learning opportunities for young people”.
 

A change of emphasis?
This objective to ‘strengthen the evidence base’ of open access youth work is carried out by the YIF Learning Project, led by NPC and the Centre for Youth Impact. Its tone and approach are encouraging, and some of the significant concerns of the youth work field have been taken on board. This is demonstrated both by a language of learning and openness, and an emphasis on collaboration with young people, practitioners and youth work organisations. The principles of the YIF Learning Project laudibly include:

• An explicit ‘move away’ from ‘blanket outcomes measurement’, acknowledging the problematic nature of measurement in open access settings;
• An emphasis on working with young people, youth workers, and organisations, rather than imposing evaluation mechanisms without consultation;
• A strong commitment to the importance of good quality practice;
• A recognition of the importance of long-term trusting relationships, of enjoyment, and of connection to community;
• An understanding that youth work’s impact differs between participants, that it may well emerge over a long period of time, that it ebbs and flows, and that it is often difficult to disentangle its specific contribution.

 
 
Dilemmas
The YIF evaluation approach is more closely aligned to youth work’s approaches and methodologies than it might have been, and this is great to see. And yet, there is still a sense that it attempts to ‘measure the unmeasurable’. As I write this, I imagine the weary sighs of colleagues in the youth impact world; however much they take on board youth workers’ views, it is never enough to stop us complaining! None of what follows is intended to criticise for criticism’s sake, or to take away from the respect with which the Centre for Youth Impact (in particular) has treated those of us who are critical of the very tenets of the youth impact agenda they were set up to promote. The following are five dilemmas that are important to address:
 
1) Despite moving away from ‘blanket outcomes measurement’, quantitative outcomes measurement continues to play a central role. Given the tendency that ‘what gets counted’ is too often the only thing that ‘counts’, how can the project guard against the preference for more structured, time-limited, ‘project-based’ youth work (that is easier to ‘measure’) over informal, open-ended, open access practice? How can the group processes that are central to youth work be recognised, when it is individual change that tends to be measured?
 
2) What are the dangers of standardising and quantifying ‘youth work quality’ and ‘young people’s views’, of inventing new tools (or importing them from other countries), and of engaging private sector consultancies and agencies to do this work?
 
3) It is inevitable that evaluation – especially on behalf of a funding agency – will affect practice, including in unintended ways. How much of a ‘data burden’ will be created for organisations? Will they really feel free to share their experiences, reservations, and honest reflections?
 
4) Can evaluation be separated from top-down performance management, judgement, comparison and control? Measurement changes how practitioners are perceived, and how they perceive themselves in relation to their work. How can data be used for collective learning without it also being used as evidence of ‘success’ or ‘failure’ by individual practitioners and organisations, and even by the field of open access youth work as a whole?
 
5) How can ‘footfall’ and other data be collected without unacceptable levels of surveillance, and breaches of confidentiality about young people’s whereabouts and their activities? How can the most marginalised young people, many of whom are (rightly) suspicious of authorities and institutions, be assured that their privacy is respected?
 
 
So what? And what next?
The current approach to evaluating the Youth Investment Fund demonstrates thoughtfulness and attention to the special characteristics and challenges of open access youth work. As a result, the experiences of young people and youth workers funded by this scheme will be more meaningful and less onerous than they would have been under a more prescriptive top-down approach. The YIF Learning Project goes some way towards challenging dominant approaches to impact measurement. Yet in other ways it is reinforcing the status quo: continuing to prioritise the measurement of individual change, converting qualitative elements of youth work (its quality and young people’s experiences) into statistics, and aiming towards a financialised ‘value for money’ analysis.
 
Ultimately, without questionning the broader context - the basis on which measurement is still preferred by most funders and governments, as a neoliberal tool of governance and control – many of these problems remain intractable. Moving beyond such dilemmas, then, is not merely a matter of creating more congruent impact tools, reducing the data burden, and involving young people and practitioners in the process (important though all of these things are). It requires imagining meaningful evaluation beyond a focus on outcomes and measurement, thinking seriously about the social and political purpose of youth work, and the role of young people in creating change. It involves working with others - beyond the youth sector and beyond our national and regional borders - to challenge the global dominance of finance and investment logic in activities that hold to a different version of ‘value’. While such aspirations may seem momentous, there is nothing to stop us dreaming of a different world, and doing what we can to make it real in our everyday lives.
 
Tania de St Croix has been a youth worker for over 20 years, and is an active part of ‘In Defence of Youth Work’ (IDYW); she thanks IDYW colleagues for helpful feedback on this blog post. She is a lecturer at King’s College London. Her book, ‘Grassroots Youth Work: Policy, Passion and Resistance in Practice’, was published in 2016, and her forthcoming research project is entitled ‘Rethinking impact, evaluation and accountability in youth work’.