One of my roles as ‘curator’ of the Tamarack Institute’s Evaluating Community Impact work is to track and share ideas and methodologies that community changemakers might find useful in their work.
Over the next six months, I will focus on evaluating systems change and social change. Innovators all over the world are focused on reforming or transforming systems, whether they be related to energy, child protection, ecological education, economic, social systems, or (more likely) a mix of all them.
Why? Because in order to make progress on tangled up social, economic and environmental challenges, we need to change the deeper systems that underlie – even produce – those challenges in the first place. As Karen Pittman, CEO of the Forum on Youth Investment, noted at a Tamarack event in 2015:
"Programmatic interventions help people beat the odds. Interventions that change the systems in which people live seek to change their odds."
The field of evaluation has not kept pace with community builders’ shift from the programmatic to the systemic. Mainstream evaluation practice emerged in the post-war period, when the focus was on assessing narrowly designed, tightly bounded programmatic interventions focused on improving the lives of people. Evaluators have succeeded in producing a broad and sophisticated field of practice. However, much of what they have produced is incapable of dealing with the demands of evaluating changes in systems. Some of the more challenging ones include:
- Accurately describing the ‘system(s)’ people would like to change
- Tracking diverse, often spontaneous, and almost always evolving interventions
- Capturing the intended and unintended effects of our efforts
- Assessing the extent to which changes in systems are due to innovators activities rather than other factors
- Authentically and ethically integrating people’s diverse perspectives on the systems we are trying to change and how they perceive those changes
- Paying attention to how power and privilege shape evaluation and learning
- Understanding what constitutes ‘rigorous design and methods’ in such complex, messy work
In the best cases, traditional evaluators, practitioners and funders clumsily try to adapt conventional evaluation practices to fit systemic work. In the worst cases, “” force innovators into programmatic evaluation boxes. This means much of their work remains invisible, the evaluation feedback unhelpful, and innovators are discouraged from trying to do systems work in the first place.
Happily, evaluators are busily self-correcting. They continue to pump out an impressive array of new concepts, methods and practices. Follow the work of Center for Evaluation Innovation, www.better Evaluation.org, or the American Evaluation Association’s Eval Talks and do your best to keep track of everything you see.
Over the next six months, I will highlight some of these important contributions on behalf of the Tamarack Institute. This will include interviews with world-renowned evaluators (i.e. Meg Hargreaves, on Planning a Systems Change Evaluation, and Ricardo Wilson-Grau on Outcome Harvesting), writing blogs that review new resources (e.g. The Tyranny of Metrics), the preparation of a paper on a result framework for systems change, and the delivery of a two day workshop organized around leading edge methodologies. These will be shared on the Tamarack website. The exploration will not be exhaustive, but instead focus on topics that respond to the many questions and concerns surfaced by Tamarack learners over the years.
The most exciting development of all of these will be watching, learning from, and sharing insights on a ground-breaking new professional development program offered by Simon Fraser University Continuing Studies: the Evaluation for Social Change and Transformational Learning Certificate. This online program, which begins this autumn, takes some of the toughest challenges of evaluating social and system change efforts head on in a half dozen courses:
- Transformative Learning
- Complexity & Systems Thinking for Evaluation
- Evaluation, Leadership and Learning
- Designing for Transformative Learning
- Data Analysis, Interpretation and Communicating Finding
- A Capstone Project
A signature feature of the Certificate is a commitment to tapping into and empowering different ways that diverse people understand and assess the world we are trying to change. This includes a special emphasis on Indigenous ways of knowing, whether the focus is on a simple program or in implementing 94 Calls to Action proposed by the Truth and Reconciliation Commission.
The Certificate is a bellwether development in the field of evaluating social and system change. I’ve enjoyed working alongside a talented group of innovators, evaluation practitioners, and community leaders in the development phase. Michael Quinn Patton, who I consider the premiere thinker on the topic, assisted in the design and it is offered by one of Canada’s most community-grounded post-secondary institutions with deep expertise with online and professional learning. If anyone is looking for a place to see how many of the leading edge evaluation concepts and practices come together, it will happen in this program.
It is an exciting time in the emerging field of evaluating systems change. It will be pleasure to watch it more closely over the next six months.