My just-released paper, Shared Measurement: The Why is Clear, the How Continues to Develop acknowledges the importance of shared measurement as one of the five conditions of Collective Impact. As noted by Kania and Kramer, “Collecting data and measuring results consistently on a short list of indicators at the community level and across all participating organisations not only ensures that all efforts remain aligned, it also enables the participants to hold each other accountable and learn from each other’s successes and failures.”
In observing the field of Collective Impact, while the “why” of shared measurement is clear, the practice of shared measurement, and the establishment of shared measurement systems that engage change makers in sense-making around that data, is still developing. Five shared measurement challenges that, if not handled well, can weaken a group’s ability to evaluate and manage their Collective Impact effort include:
- Ensuring that shared measures are organized in a way that reflects the group’s evolving strategy or theory of change – Many Collective Impact participants and evaluators often rush to select the best measures for their effort before they have fully developed their common agenda and strategy. As a result, their conversations often spin around issues such as which indicators they could collect – rather than which ones they should.
- Distinguishing between shared outcomes, measures and measurements – Collective Impact participants often get confused about the distinction between shared outcomes, shared measures and shared measurement. As a result, they often invest more time and attention on shared measurement than is necessary.
- Creating good sense-making and decision-making processes – Collective Impact participants can become consumed with the production of data. In fact, they can become so consumed that they forget to pay attention to developing robust processes that (a) make sense of data, and (b) use it to help make decisions about the Collective Impact strategy and operations.
- Employing a mix of “big design” and “agile” approaches to the development of shared measurement systems – One of the most consistent pieces of advice offered by advocates of shared measurement systems is that organizations should develop their approach. The limitations of this approach are: it can be expensive and time-consuming; and, it can result in unwieldy processes that are difficult to adapt once the final design is complete and implementation begins.
- Acknowledging, monitoring and responding to a variety of perverse behaviours that often emerge with measurement processes – Although advocates and practitioners of shared measurement are clear about the possible benefits of the practice, they often are unaware of the perverse consequences that may accompany it. Collective Impact participants need to: beware that perverse behaviours are likely to emerge; vigilantly monitor their work to spot these problems early; and, once discovered, take remedial action to eliminate them.
In spite of these challenges, evaluation and shared measurement form a cornerstone of effective Collective Impact approaches and, as the field of practice in shared measurement continues to evolve, we need to be willing to develop and adapt our evaluation frameworks, methods and measures to more effectively inform our practice.
Learn More:
- Read Shared Measurement: The Why is Clear, the How Continues to Develop by Mark Cabaj
- Join Mark Cabaj and Liz Weaver for Evaluating Community Impact, a 3-day learning experience November 14-16, 2017 in Saskatoon, SK
- Learn more and find practical resources on Collective Impact and Evaluation at tamarackcommunity.ca
