YPQI Pilot next steps - how can we achieve quality at scale?

Over the past three years, the Centre for Youth Impact has been piloting an innovative approach to continuous quality improvement across the UK through the Youth Programme Quality Intervention (YPQI), with thanks to support from The National Lottery Community Fund. Through the YPQI process, practitioners are empowered to take ownership of the behaviours, processes, and data that contribute to ongoing quality improvement and high quality provision for the young people they support through peer observations and improvement planning. For more background information about the YPQI, click here.

Over the course of the pilot, between 2018 and 2021, the Centre has mapped, adapted and developed YPQI materials and training for the UK youth sector, delivered extensive training opportunities for practitioners at over 100 organisations, and built partnerships to start embedding continuous quality improvement. Like many others, we also significantly adapted our delivery and support offer in light of the Covid-19 pandemic. Alongside this, we’ve undertaken dedicated research to explore the opportunities and barriers to embedding the YPQI in practice. 

 

Key lessons from the pilot

The pilot has provided a strong base of learning from which we are building our next stage of work. Here are some of the things that we’ll be thinking about as we move forward:
 

Suitability

• Practitioners and stakeholders feel there is a need for a structured approach to improving the quality of youth work practice in the UK.  Whilst the concept of the YPQI is perceived to meet this need, and it is generally valued by participating organisations and stakeholders, the full process requires either flexibility or adaptation (or a combination of the two).
• Whilst need might be recognised, the appetite for quality improvement and measurement is mixed, according to stakeholders.  Whilst youth workers really care about doing a good job, capacity and resources for measurement and improvement are limited.
• Youth organisations’ motivations to use the YPQI process are primarily ‘internal’ (e.g. wanting to improve the quality of their work for young people) but ‘external’ motivations such as demonstrating the quality of their work and attracting funding are also important.
• The PQA is seen as a way of defining what good quality youth work looks like, which can help to define or demystify youth work to stakeholders. This includes differentiating youth work from other ‘work with young people’.

Value, impact and effectiveness

• Whilst participating organisations felt it was too early to describe the impact of the YPQI, they believed it was valuable. Examples included: better quality data to inform development; more open conversations and critical thinking about quality; staff development; and ultimately improvements to practice. 
• Stakeholders believed the YPQI was likely to be effective in improving quality because of a clear focus on what good youth work looks like (i.e. the PQA framework) through the process of reflective practice.

Implementation

• In practice, there is a high level of adaptation of the YQPI process in youth organisations that are using it. Participants tailor the process to fit their organisation, which commonly includes some simplification (e.g. focusing on parts of the PQA framework rather than the whole). Whilst this improves take up, it also presents a risk to ‘fidelity’ and the evidence base that supports the impact of the YPQI on outcomes for young people. 
• Barriers to implementation relate to: suitability (is the PQA framework representative of our work?), process (e.g. is it practical to organise the training and meetings required?); capacity (have we got time and headspace to do this?); and feeling overwhelmed by the content and process in the early stages (how do we get our heads around this, and align it with other organisational systems and processes?). Those who left the process felt the YPQI was too time consuming and/or complex and that the framework was not completely aligned to their work.
• There are mixed views about the types of provision for which the YPQI is suitable based on both perceptions of how well the framework represents the type of provision (e.g. detached vs buildings-based), and how feasible it is to observe the provision type (e.g. open access with multiple activities vs structured provision). 
• Common enablers of YPQI implementation include: strong leadership (by both senior leaders and staff), flexible approach to delivery, a ‘piloting’ approach to test what works, having a focused plan (e.g. on which areas of provision to focus rather than everything at once), embedding the PQA framework in other aspects of work such as supervision, a learning culture, and flexible support from the Centre for Youth Impact.
• Training and resources were also viewed as very high quality.

 

Key Lessons from the YIF

Alongside the pilot (which any organisation could apply to take part in) we also integrated key aspects of the YPQI into the Youth Investment Fund (YIF) learning project, which ran from 2017 to 2020. Youth organisations funded by the first Youth Investment Fund were able to opt into using the core YPQI tool (the Programme Quality Assessment, or ‘PQA’), and collecting data from these organisations enabled the research team to disaggregate data across the YIF cohort to understand how and why outcomes for young people varied. Our YIF findings indicated that:
 

• Young people in higher quality provision (as measured by the PQA) made greater progress on outcomes related to social and emotional skills, social connectedness and well-being than those in lower quality provision. Young people’s feedback was also more positive in higher quality settings.
• Young people who faced particular disadvantage made greater ‘gains’ in their social and emotional learning in higher quality settings: in fact, these young people ‘overtook’ the SEL growth of more advantaged young people in medium quality settings. The quality of settings was also closely related to young people’s opportunities to feel heard, show leadership, and affect change. This is the ‘equity effect’ of high quality provision. 
   

This is the first time that such data has been collected, analysed and shared in the UK, and it demonstrates that there is real potential for the YPQI to contribute to a stronger evidence base for work with young people in the UK, and for youth practitioners and organisations to use the YPQI as part of their ongoing evaluation practice and practice development in a way that supports better outcomes for young people. We are really excited by these findings.

However, as our experience and the implementation research indicates, the core YPQI pilot demonstrated that consistent and ‘faithful’ implementation of the YPQI process, at scale, is challenging. Outside of the YIF, we have not been able to generate sufficient data to further build the supporting evidence base. Alongside the barriers noted above, we believe that this is in part due to a lack of supporting infrastructure; for example, some ambiguity around different approaches to 'quality', and limited of prioritisation of and advocacy for continuous quality improvement that is driven by data.  However, challenges at organisation level (e.g. limited capacity) have been further aggravated by the Covid-19 pandemic.

Next steps 

Our implementation research suggests that ways to improve take-up of the YPQI could include more intensive support on the ground from practitioners with experience of delivering the YPQI; simpler and clearer messaging; demonstrating success through examples; external recognition of the value of the YPQI (e.g. by funders); peer support; a ‘total systems approach’ to align quality practice, clarifying the unique contribution of different approaches that already exist within the youth sector; improved evidence on the ‘quality to impact’ link (i.e. building on the YIF findings);  and embedding the PQA framework in the wider system, e.g. an improved focus on social and emotional learning (SEL) training for youth workers.

In order to move forward, and to ultimately build a better evidence base for the relationship between quality, outcomes, and equity for young people in the UK, we believe that there are a number of priority areas we must focus on. We are delighted to have received funding from the Esmée Fairbairn Foundation to support this next phase of work, which will initially focus on three key areas:
 

• Infrastructure: scoping and developing regional communities of practice, alongside a national community of practice, that is focused on quality. These communities will offer peer support and opportunities to share data, and ‘low stakes accountability’ for YPQI implementation for youth practitioners.
• Advocacy: building our shared understanding of the relationship between equity, quality youth work practice and SEL, drawing on and promoting what is already known from previous studies alongside a vision for what the UK youth sector can achieve from future research and an increased evidence base.
• Training and workforce development: working closely with key partners and stakeholders to explore how we can embed core quality principles in youth work training for new practitioners, alongside additional developmental training opportunities for youth work practitioners. We will also continue to support organisations and practitioners who took part in the YPQI pilot and wish to continue with their implementation of the programme.

 

We believe that this work will lay the foundations for future efforts focused on building research and evidence and organisational behaviours for continuous quality improvement.
 

Finally, we will also be using this time to reflect on and respond to key questions emerging from the YPQI pilot, which include:

• The tension between focusing on internal learning and reliable external assessment of quality;
• How the PQA can be used as a framework for practice and/or measurement;
• Making core components of the whole YPQI clearer and establishing what ‘fidelity’ means within the context of YPQI implementation; and
• A revised set of hypotheses for what PQA (and other) data can tell us about quality, outcomes, and equity for young people.

 

Transitioning from a pilot to a short and long-term offer for participants

For the time being, we will focus resources on supporting organisations who have already undertaken training and begun implementing the YPQI as part of the pilot programme. However, as noted above we hope to provide opportunities for other practitioners to get involved through our workforce training and development offer, drawing on the principles of high quality social and emotional learning practices, and ongoing cycles of reflection, planning, and improving.
 

Over the next 9 months, we will also be developing a longer-term sustainability plan for supporting practitioners and organisations with ongoing quality improvement further into the future.
 

Thank you

Finally, we would like to extend our gratitude to all those who have taken steps towards joining us on this journey towards more meaningful quality measurement and improvement. There is much work to be done, but we are excited to move into the next phase of delivery in partnership with you all. We also thank The National Lottery Community Fund for all its support in the pilot to date, and all of the practitioners who have taken time to share their feedback and reflections with us. 

We will share more detail on our specific plans soon. In the meantime, if you would like to find out more, please get in touch via ypqi@youthimpact.uk.