The Latest

Contribute. We love to hear your thoughts, your musings and your latest work. Please share with us!
Write a post

Principles-Focused Evaluation: Newest Approach for Evaluating Complex Change Efforts

Posted on December 11, 2017
By Mark Cabaj

Principles Focused Evaluation Book Cover-4One of the toughest challenges for social innovators and evaluators is to describe the “it” they are trying to evaluate. While they typically have a general idea of the outcome they would like to see (e.g., an end to homelessness, addressing mental health issues, a feeling of inclusion), they often struggle to lay out the pathway(s) to get there. This makes it difficult for both parties to land on questions, indicators and/or methods around which to build an evaluation design. 

Evaluators have tried to help. In the early days, they created “evaluability assessment”, a method for assessing whether the interventions to be evaluated met the conditions required to put together an evaluation, such as clear goals, measures of success, and maps of how their activities would lead to these outcomes. When policy makers and program designers struggled to meet these conditions, evaluators rolled up their sleeves and got into the business of intervention design. Their inventions include logic models, results chains, change pathways, theories of change and – more recently – hypotheses of change.

While each of these tools are useful in specific situations, they have not been able to accommodate a simple yet critical reality that the thinking and action of many would-be change makers are guided by principles rather than logic models. These include:

  • Principles that they deem important for the effectiveness of their interventions (e.g., “Housing First”, which means providing vulnerable persons living on the street with housing before anything else); 
  • Principles that guide the innovator’s efforts when specific designs need to vary from place to place or adapt to an ever-evolving environment (e.g., “Adapt models to reflect local context”); and,
  • Principles that reflect deeply held values and beliefs about the proper way to live and behave (e.g., “Nothing about us without us,” which means people with lived experience should be deeply involved in shaping and assessing any intervention that affects their lives).

Yet, for all their power and influence, principles are often implicit.  They live in the shadows of social innovation.

In his latest book, Principles-Focused Evaluation: A Guide, Michael Quinn Patton pulls the principles-focused approach out of the shadows and crafts a compelling and coherent approach for integrating them into evaluation practice. The book is organized into five sections:

  • Section 1 – Foundations: This section provides an exploration of different types of principles, why they matter, and the niche for principles-focused evaluation in the larger evaluation field. 
  • Section 2  Guide: This section reviews the five types of criteria for effective principles.
  • Section 3 - Exemplars: This section offers a scan of principles-based interventions guided by principles-focused evaluation.
  • Section 4 - Principles for Evaluations and Evaluators: This section reviews issues related to integrating a principles-focused approach into different methods and indicators.
  • Section 5 - Tools and Checklists: This section offers a variety of helpful techniques for planning and implementing principles-focused evaluation.

As in all his previous books, Patton illustrates his ideas and practices with an impressive number and variety of anecdotes, vignettes and stories. These focus on different domains (e.g., poverty, homelessness, agriculture, etc.) and operate on different scales (e.g., organizational programs to international). All of them are informative and demonstrate the value of principles-focused evaluation in a wide range of contexts. The Vibrant Communities case study, for example, will be of interest for people interested in poverty reduction, country-wide initiatives, and Canadian examples.

With principles-focused evaluation, Patton has (once again) made a significant contribution to the field of evaluation, as well as to the theory and practice of social innovation. Principles-focused evaluation is a game-changer for social innovators, evaluators, policy makers and funders who are interested in making – and evaluating – progress on the tough economic, social and environmental challenges of our time.

Learn More:

Evaluating Community Impact, Evaluation, Mark Cabaj

Mark Cabaj

By Mark Cabaj

Mark is President of the consulting company From Here to There and an Associate of Tamarack. Mark has first-hand knowledge of using evaluation as a policy maker, philanthropist, and activist, and has played a big role in promoting the merging practice of developmental evaluation in Canada. Mark is currently focused on how diverse organizations and communities work together to tackle complex issues, on social innovation as a "sub-scene" of community change work, and on strategic learning and evaluation.

Related Posts

An Introduction to Jean-Marie Chapeau

Living History: A Developmental Evaluation Tool

Why do we still think youth aren’t interested in program evaluation?