Evaluating Systems Change Results: An Inquiry Framework

Posted on December 18, 2018
By Mark Cabaj
What do you mean on cementAt Tamarack’s Community Change Institute in Vancouver in 2015, Karen Pittman, CEO of the Forum on Youth Investment, shared a noble prize worthy piece of poetry: Programmatic interventions help people beat the odds. Systemic interventions can help change their odds.

The crowd roared with approval.  Karen captured an idea that was now increasingly mainstream in social innovation circles: in order to make deep and durable progress on tough economic, social and environmental issues, we must change the systems underlying those issues, the systems that keep them in place.

While the idea of systems change is clear, the practice is not.  The same month that Karen spoke at the Tamarack Institute, Donna Podems, an experienced evaluator, described just how difficult it is for social innovators and evaluators to describe what it looks like in their context:

I was asked to work with innovators in the national health program of an African country. When I started working with the group, they said, ‘We aim to shift the health system.’ After listening for a few hours, I said, ‘Honestly, I have no idea what you are doing, or what you are trying to achieve … and I haven’t a clue how to measure it. I don’t understand what it means to “shift the health system.”’ And they looked at each other and burst out laughing and said, ‘We have no idea either.’ [1]

Developing a clearer sense of what we mean by ‘change’ and results’ in systems change efforts is a high stakes challenge.  We need it to sharpen our thinking about strategy. We need it to develop and track indicators of progress so that we can learn from our efforts. We need it to be able to communicate our work amongst our allies and those whose support us as we seek.

Thankfully, there are a growing number of excellent resources on defining, planning and evaluating systems change out there. Yet, because getting our head around what we mean by system change is important, one more won’t hurt. Evaluating Systems Change Results: An Inquiry Framework, describes three key types of results that social innovators and evaluators should consider ‘mission-critical’ in their work.

[1] Patton, Michael Quinn, McKegg, Kate, Wehipeihana, Nan. 2015. Developmental Evaluation: Principles in Practice. Guilford Press: New York. Page. 293.

Learn More:

Evaluating Community Impact, Evaluation, Systems Change, Blog

Mark Cabaj

By Mark Cabaj

Mark is President of the consulting company From Here to There and an Associate of Tamarack. Mark has first-hand knowledge of using evaluation as a policy maker, philanthropist, and activist, and has played a big role in promoting the merging practice of developmental evaluation in Canada. Mark is currently focused on how diverse organizations and communities work together to tackle complex issues, on social innovation as a "sub-scene" of community change work, and on strategic learning and evaluation.

Related Posts

How we can protect the social role of government: What is social procurement?

How to connect: A visit to Davos prompts questions and connections

Three big ideas for 2024