Measurement Tools Spotlight - YMCA Heart of England

YMCA Heart of England, formed in 2019 from the merger of YMCA Birmingham and YMCA Coventry and Warwickshire, is one of the larger organisations within the YMCA Federation. With nearly 100 staff, the organisation’s offer includes supported and independent accommodation, a youth service, a community hub, and a nursery. Among its other projects, YMCA Heart of England runs Coventry Youth Service. The service, based in Willenhall, one of the most deprived areas in England, has three full-time members of staff, two volunteers, and a team of Youth Ambassadors. There is a youth club, which welcomes about 45 young people per week, as well as sports, social action projects, and community work. Attendees are often from marginalised socio-economic backgrounds or have experienced adverse childhood experiences (ACEs). The youth service also offers referral-based interventions with young people through mentoring and life skills in schools and other establishments.

YMCA Heart of England is moving towards a digital approach to data and evaluation, and is using the College’s free measurement tools designed to understand and measure young people’s gains in socio-emotional skills, the quality of provision to support socio-emotional skills, and the level of young people’s engagement in this provision. Each service within the organisation has been collecting a wide variety of data in different ways to understand the impact of their work.

It was folders, pens, paper, scribbles, Excel printouts… case studies, testimonials, feedback on post it notes at the end of the session, interviews…

YMCA Heart of England worked with a researcher to determine which of the College’s measures would be most appropriate to use to measure the impact of Coventry Youth Service. The organisation has used the Practitioner Observation Tool[1] (POT), the Youth Engagement Survey[2] (YES) with around 30 young people who engage regularly with the Youth Service, and the Young People’s Survey[3] (YPS) with a further 10 young people.

Staff emphasised the importance of clearly explaining the purpose of the surveys to the young people and encouraging them to complete them as fully and as honestly as possible. This helps ensure the tools capture accurate data on the provision’s impact – including what works and what doesn’t – to inform improvement.

We're really honest [with young people] about what the surveys are meant to be for. We say that [funders] want to know exactly how you feel about this service; how you want it to be better. So, if you put all 10s, they're going to say... we don't need to fund you anymore... We actually try and challenge them and say, ‘How do we get better?’

Staff welcome the measures’ inclusion of clear questions on topics which map clearly to the organisation’s theory of change (empathy, emotional management, and problem solving), values how straightforward it has been to cascade the use of the measures across the staff team.

Having externally validated questions that young people can understand ensures the data is meaningful and can be used to target provision accurately. And to ensure all young people fully understood what was being asked of them, staff at YMCA Heart of England provided one-to-one support to help young people interpret and complete the survey questions.

Gathering data from young people helps the organisation tailor their services more closely to young people’s needs and staff found young people were motivated to complete the surveys when they understood their data would be used to improve the Youth Service. Although additional incentives - like pizza - to help maximise response rates didn’t hurt either!

We're actually getting more of a bespoke service for that young person, and we're delving into their journey of their experience within our services.

 

Use of the College’s measures and its data portal[4] gives YMCA Heart of England a safe place to store data that multiple staff can access. Staff expect the portal and measures will benefit the organisation over the longer-term by giving the organisation a place to collect and visualise data that can inform its bid-writing, funder reports, strategic plans, and annual reports.

It has upskilled our knowledge and experience in collating data… The bar is raised and everybody feels comfortable in doing it. It's not just a manager job, not just one person job, everybody can get involved.

 

YMCA Heart of England’s experience with the measures has encouraged them to start using them to assess the impact of its wider services. They’re excited about the lessons that could be learned from using the College’s full range of integrated measures, including its Quality Practice Tool[5] (QPT), because the measures are designed to be used in combination to build an overall picture of youth provision. The organisation hopes that using the measures will be an important step in simplifying its data collection processes and integrating these processes across the organisation.

It makes my life a lot easier having the systems in place. You can just input the questions and data and it's done for you.

--

Please visit our measurement hub to find out more about our tools and how to use them.

 

[1] The POT is completed by practitioners within any type of provision. It captures their observations of specific behaviours of young people which are linked to socio-emotional skills.

[2] The YES is a survey completed by young people to assess their mental engagement (e.g. enjoyment, inclusion, attention, and voice) during provision.

[3] The YPS is a survey completed by young people to assess young people’s functional skills (i.e., how they feel and behave in life) in terms of emotion management, empathy, problem solving, initiative, teamwork, and responsibility.

[4] The Youth Impact Portal is a free to use, accessible web-based platform for collecting, managing and viewing data in relation to measuring the impact and quality of youth provision.

[5] The QPT is a measure of the quality of provision, focusing on staff or volunteer practices that develop socio-emotional skills. It can be done as a self-assessment, a peer or manager assessment, or by a trained external observer.