Community builders eager to make progress on complex issues must move beyond projects and programs and seek instead to change the systems that underlie the challenges. As Karen Pittman, CEO of the Forum on Youth Investment noted in a Tamarack event in 2015:
While programmatic interventions help people beat the odds, systemic interventions change their odds.
The increased focus of social innovators on changing systems must be matched by a an increased focus by evaluators. This is anything but straightforward. The concepts and methods of traditional evaluation emerged when the emphasis was on assessing projects and their effects on people. They are often unsuitable for the more complex, larger scale and dynamic work of systems change.
Happily, the field of evaluation is evolving. There are plenty of excellent examples of evaluating systems changes and helpful sets of principles, frameworks and methods. There is, however, still plenty more to do: most would-be changemakers still struggle to get a handle on the results and learnings of their work.
Tamarack is devoting the second half of 2018 to explore two important developments in the emerging field of systems evaluation. These include: (a) how to plan an evaluation of systems change and (b) some emerging methodologies for evaluating systems change.
Evaluating Systems Change: An Inquiry Framework
Social innovators, evaluators, and community changemakers are increasingly focused on changing complex systems, but often struggle to describe either the systems itself or what they hope to achieve. This paper is designed to give clarity on how to approach the evaluation of systems change and provides three types of results that social innovators and evaluators should consider “mission-critical” to their work.
In 2018 Tamarack hosted two webinars on Evaluating Systems Change. Look out for more resources as we continue to explore this topic!
In this webinar, Meg Hargreaves and Mark Cabaj explore the ideas and steps required to properly evaluate ‘system changes’ that emerge when tackling complex issues. Using her experience in systems change evaluation, and her publication "Evaluating Systems Change: A Planning Guide", Meg provides a foundation for understanding and approaching this important topic.
For more on Planning Systems Change Evaluation, see Evaluating Systems Change: A Planning Guide
Almost anyone who is interested in making progress on tough economic, social and environmental issues is committed to changing the system. Yet, many social innovators — and the evaluators that support them — struggle to describe what systems they want to change and what they mean by ‘results’. This webinar explores an archetypical results framework that can be used to inform the thinking of any effort to ‘change’ systems.
Outcome Harvesting. Read Mark’s review of Ricardo Wilson-Grau’s upcoming book on Outcome Harvesting, a systematic, and refreshingly straightforward methodology to track the changes that emerge out of complex change initiatives.
The Tyranny of Metrics. Read Mark’s review of the provocative new book that argues that our obsessive preoccupation with quantifying the performance of complex systems – in philanthropy, the military, education and health care – distorts our understanding of what is and is not working, requires an excessive investment of time and energy and may even undermine our effectiveness.
Here are a number of resources that provide a summary of principles and methodologies that social innovators and evaluators may find useful in their efforts to change systems:
Systems Change: A Guide to How to Do It - A very accessible, introductory level guide to explore the basic ideas of systems change prepared by the Lankally Chase Foundation.
Evaluating Systems Change: A Planning Guide - A systematic approach to planning an evaluation of systems change initiatives.
Evaluating Complexity: Propositions for Improving Practice - A list of ten principles and associated methods developed by the Foundation Strategy Group (FSG) for evaluating complex change initiatives, including systems change.
Systems Concepts in Action: A Practitioners Toolkit - A first rate resource that explores the application of systems methods to investigate, evaluate, and intervene in complex and messy situations.