The power of impact, the impact of power

The place of power in evaluation is an interesting question to ponder: it is somehow everywhere and yet simultaneously nowhere. Power relations could arguably be the single biggest influence on how an evaluation is designed, undertaken and experienced: power often decides the questions that an evaluation is seeking to answer, and how it goes about answering them. Yet in the results, or the write up – particularly of large scale, ‘heavyweight’ impact studies and service evaluations, these power relations are rarely explicitly acknowledged. This is despite acknowledging that many go into the evaluation ‘profession’ precisely because they wish to amplify the voices of those who are seldom heard or listened to, and to redress some of the power imbalances that persist.

Similarly, where and how we perceive power to be located and ‘held’ is a major determinant of how we define not just ‘impact’ but even ‘change’. Who gets to define what ‘good’ looks like, or what constitutes ‘progress’? Perhaps the most common example of this relates to ‘empowerment’ as a focus for provision. Implicit in the aim of empowering young people is a recognition that a power imbalance is present, but it also says something a little more challenging about how power flows and may be granted or ‘bestowed’ on one community by another. These more contentious questions about power can be hard-wired into theory of change (both the process and the output), and ignored or even exacerbated by evaluation. We should question whether anything other than young people’s voices can tell us about the extent to which they ‘feel empowered’.
 
Power is also intimately woven into how we consider and value evidence: what we decide to share, and what we decide to hear and act upon.
 
We often hear that for many people, stories are more powerful than numbers – “everyone loves a case study”.  To others, the words of individuals can appear to have very little power: greater power seems to reside in assimilating voices, translating them into numbers and ironing out the differences.
 
In either case the picture can become more complicated. The power that stories bestow has the potential to shift when third parties select some to be retold and disregard others, and appropriate these stories and use them to meet their own needs.  Similarly, the people doing the assimilation, translation and ironing required to provide neat statistics represent a further locus for the power bestowed by information.
 
One place where power is very explicitly referenced in evaluation is the concept of ‘statistical power’. This refers to the likelihood of over-looking the effect of an intervention because there was not enough data to distinguish between the effect just happening by chance. This is intriguing to me: it’s a case of there being power in numbers (that is, the more data, the more confident we can be about the finding) but yet little power in individual experience. The risk is that individual experience can be literally ‘invalidated’.
 
It is with these thoughts in mind that we’ve decided to focus on the power question for our annual conference this year. It would be relatively easy – and indeed, I’ve had this conversation many times – to ascribe malign intent to evaluation; to suggest that the act of evaluating disempowers or actually devalues that which we’re trying to capture. I agree that this is certainly a possibility, but I don’t think it’s inevitable. Evaluation can be as ‘empowering’ as disempowering, and as powerful as it can be toothless. Fundamentally, evaluation is nothing without human intent and human interaction, even though many evaluation designs seek to limit or ‘control for’ this. And so, we need to talk about it more openly, to consider how power and evaluation are experienced, and how it might be different.
 
And that’s what we hope our conference will do. We will hear challenge about whether evaluation will always be a tool of the powerful, and whether there’s an opportunity to reclaim it – as many are already trying to do. We’ll explore whether a focus on the quality of experience might helpfully shift the line of sight away from our power to create impact, and towards our power to create the right conditions in which communities create impact for themselves. We’ll consider the power of data, funding, the collective, and lived experience. We look forward to seeing you there!

Buy tickets and view the programme for The Gathering 2018 here.