Longitudinal perspective on the impact of youth provision: easier said than done?

The Centre for Youth Impact recently hosted its fourth annual Gathering, and a clear theme that resonated throughout the conference was the need to invest in and improve longitudinal research in the youth sector. This blog, written by Sarah Williams (the Centre's Research and Learning Assistant), explores the challenges of  getting good quality longitudinal data and suggests some next steps on where to go from here.

It is widely held that the most significant impacts of youth work in particular may be experienced long after a young person has directly come into contact with an organisation or service, extending well into their adulthood. However, surveys that are premised on linear, short-term changes remain the dominant form of evaluation. Many and diverse voices have been calling for approaches that apply a greater focus to understanding longer-term change.
​
The challenge of obtaining good quality longitudinal data
Predictably, there’s not a cheap or easy short-cut to gathering good quality evidence on the long-term impact of youth provision. Although there have been positive steps to create longitudinal data sets from existing administrative data (e.g. NPC’s Data Labs), longitudinal studies tend to be complex, expensive and time-consuming, often requiring large sample sizes and necessarily lasting many years before the results bear fruit. For youth workers who are operating on a tight budget and with limited time, this makes longitudinal research at best difficult, and at worst impossible, to do themselves.

There are, however, a large number of existing longitudinal studies in the UK which already collect some quantitative data on youth provision, including the Second Longitudinal Study of Young People in England (LSYPE2); the Millennium Cohort Study (MCS); and the UK Household Longitudinal Study (UKHLS) (also known as ‘Understanding Society’). So, what is the potential to utilise the data captured through these surveys to determine the impact of youth provision on future life outcomes?
 
Utilising the existing data
A pioneering example of this type of research was conducted by Leon Feinstein and colleagues (2006) using age-16 data from the 1970 British Birth Cohort Study (BCS70). This research suggested a correlation between young people’s leisure activity and their later life-course outcomes, particularly in relation to their identity formation and future social mix. For example, the study found that participation in some structured youth activities at age 16 - including church groups and sports centres - was associated with reduced likelihood of social exclusion outcomes, such as depression, homelessness, or living in a workless household. On the other hand, participation in ‘youth clubs’ showed a very different picture, since these services typically attracted individuals from a lower socio-economic background, and were actually associated with an increased probability of social exclusion experiences during adulthood (such as unemployment and crime). These research findings were strongly challenged by the youth sector, and arguably, were interpreted over-simplistically in policy terms. Unfortunately, the heightened tensions that endured following the publication of the research overshadowed the potential for learning, both about the findings and the nature of longitudinal studies more generally. 
However, it remains the case that this type of analysis is crucial for enhancing our understanding of the factors that shape the learning and awareness of young people, to comprehend how youth provision can support a long-term and complex developmental process. Furthering and expanding on this type of research is critical in making the most of the longitudinal data that has already been collected on youth provision.
 
The need to improve future data sets
It has now been over a decade since Leon Feinstein’s research on the BCS70, and since then there has been a distinct lack of analysis of longitudinal surveys in relation to youth provision and youth work in particular. Whilst further analysis of this type would be valuable, there are a number of issues with the existing data that prevent longitudinal studies from reaching their full potential for the youth sector:

1. Some survey questions are highly simplistic, lacking consideration for the specific type of service or provision used by a young person or their motivations for participating;
2. The surveys often fail to capture the degree to which a young person has engaged with the provision offered to them, lacking detail on their frequency and intensity of engagement;
3. Some of the questions fail to distinguish whether the provision used by a young person took place within or outside of school.

So, where do we go from here?
Although the UK has a rich base of longitudinal research studies, the existing data on youth provision is often vague and inconsistent. As well as utilising the data we already have, advocating to amend and improve future data sets - through more refined and detailed questions - could open the door to further meaningful analysis. This is a critical step for helping to explain the wider and longer-term impacts of youth work, offering value not only to the present generation of youth practitioners and policy-makers, but also for generations to come.
This blog is very much a starter for ten on this topic, and we will be doing more work to explore how we can make the most of existing longitudinal data and improve the data in the future. We are keen to work with partners across the sector on this agenda so if you want to work with us or would like more information please contact sarah.williams@youthimpact.uk.