Tamarack's Galen MacLusky discusses the ORS Impact and Spark Policy Institute report "When Collective Impact Has an Impact", joined by Terri Akey (ORS Impact) and Lauren Gase (Spark Policy Institute)
The team from ORS Impact and Spark Policy Institute reviewed the results of 25 Collective Impact efforts including three in Canada and conducted deeper dive visits with eight sites. This webinar explores the report and the lessons learned from this extensive evaluation effort.
After the session, Terri and Lauren provided written responses to your additional webinar questions:
Question: What tools of analysis or methodologies did you use for analysis of CI successes?
We examined early changes, systems changes, and population changes. Each of these types of changes is described on page 18 of the report. Rubrics for understanding the dimensions of systems and population change can be found in the appendix (tables A4 and A5)
Question: What are funders reactions to the findings, and using mixed methods and not randomized controlled trials (RCTs) to connect the dots between activities and outcomes
We used process tracing (described on page 10 of the report and further in the appendix) to examine the relationship between the collective impact approach and its contribution to population change. This was our way to use rigorous, applied methods to answer the study questions. Given the complex, retrospective nature of the work experimental methods (like RCTs) were not feasible or appropriate. Readers/audiences seem receptive to the methods; we have not received any specific criticisms.