Evaluating Systems Change Results: An Inquiry Framework

BY: MARK CABAJ

At Tamarack’s Community Change Institute in Vancouver in 2015, Karen Pittman, CEO of the Forum on Youth Investment, shared a noble prize worthy piece of poetry: Programmatic interventions help people beat the odds. Systemic interventions can help change their odds.

The crowd roared with approval.  Karen captured an idea that was now increasingly mainstream in social innovation circles: in order to make deep and durable progress on tough economic, social and environmental issues, we must change the systems underlying those issues, the systems that keep them in place.

While the idea of systems change is clear, the practice is not.  The same month that Karen spoke at the Tamarack Institute, Donna Podems, an experienced evaluator, described just how difficult it is for social innovators and evaluators to describe what it looks like in their context:

I was asked to work with innovators in the national health program of an African country. When I started working with the group, they said, ‘We aim to shift the health system.’ After listening for a few hours, I said, ‘Honestly, I have no idea what you are doing, or what you are trying to achieve … and I haven’t a clue how to measure it. I don’t understand what it means to “shift the health system.”’ And they looked at each other and burst out laughing and said, ‘We have no idea either.’ [1]

Developing a clearer sense of what we mean by ‘change’ and results’ in systems change efforts is a high stakes challenge.  We need it to sharpen our thinking about strategy. We need it to develop and track indicators of progress so that we can learn from our efforts. We need it to be able to communicate our work amongst our allies and those whose support us as we seek.

Thankfully, there are a growing number of excellent resources on defining, planning and evaluating systems change out there. Yet, because getting our head around what we mean by system change is important, one more won’t hurt. Evaluating Systems Change Results: An Inquiry Framework, describes three key types of results that social innovators and evaluators should consider ‘mission-critical’ in their work.

[1] Patton, Michael Quinn, McKegg, Kate, Wehipeihana, Nan. 2015. Developmental Evaluation: Principles in Practice. Guilford Press: New York. Page. 293.

Learn More:

• Read: Evaluating Systems Change Results: An Inquiry Framework
• Listen to a webinar with Mark on Evaluating Systems Change: A Results-Based Framework
• Learn more about our exploration of Evaluating Systems Change

Share this article:

Change is in the Air

BY: LIZ WEAVER

November took me to Australia and New Zealand to meet with colleagues working on community change, community-led development and Collective Impact.  I was a keynote speaker and workshop presenter at ChangeFest, a gathering of 500 Collective Impact practitioners in Logan, Queensland.  Collective Impact is moving forward in Australia.  Over the four days in Logan, participants experienced and exchanged ideas, strategies and challenges.  Many of the more than 90 Collective Impact efforts currently operational in Australia shared their outcomes and learnings.  Logan Together, Collaboration for Impact, Opportunity Child and Griffith University were the hosts of ChangeFest and each are leading and supporting Collective Impact efforts across Australia. 

A major focus of ChangeFest was how Collective Impact can move the systems and policy landscape.  The four-day festival of ideas, embraced the Uluru Statement from the Heart as an important path forward for those organizations and collaboratives employing Collective Impact.  The Uluru Statement from the Heart recognized the ancient sovereignty of Australia’s Aboriginal and Torre Strait Islander peoples.  This sovereignty has not been formally recognized by the Government of Australia. 

My next stop was New Zealand, to celebrate the tenth anniversary of Inspiring Communities.  Across New Zealand, Inspiring Communities has worked deeply with citizens and neighbourhoods in the practice of community-led development.  Over the years, Tamarack and Inspiring Communities have worked collaboratively to advance the voice of citizens.  The Inspiring Communities panel at the anniversary celebration reflected on their experiences attending Tamarack’s Community Change Institute and how this experience along with the connection to Inspiring Communities has informed their work.  The Inspiring Communities anniversary celebration was not only a celebration but also an opportunity to look forward. 

Back in Australia, I also met with groups including the Northern Territories schools initiative,  Child and Youth Area Partnerships across Victoria, Family Life Australia, Beyond the Bell, the Local Community Services Association (NSW) and the Victoria Department of Health and Human Services. 

During the conversations, presentations, workshops and gatherings, a number of themes kept recurring:

• Community change needs to value and embrace Indigenous knowledge and practice: Collective Impact and community change leaders can walk alongside Indigenous leaders to forge new paths of change. 
• Community change needs to occur at many levels: Local Collective Impact efforts can make a difference in communities but sharing our experiences, challenges and learning can lead to deeper and larger-scaled impacts.
• Community impact requires investment: These investments include supporting community change leaders, capacity building, resources and time. 
• Systems impact requires capacity and knowledge: Wishing to impact systems is not enough, community change leaders need to grow their knowledge and capacity to understand how to influence policy and systems.
• Collective Impact is movement building: Collective Impact leaders need to both look inward to their work and outward to the community. 

Collective Impact, community-led development and community change are growing movements.  To scale effective and impactful change, we need to focus on people, process and most importantly impact.  Change is in the air. 

Learn more:

• Join Liz Weaver and Sylvia Cheuy at a new Collective Impact Train-the-Trainer workshop - Collective Impact: Leading Theory to Action
• Learn more about ChangeFest
• Read the Uluru Statement from the Heart
• Visit the Collaboration for Impact website
• Visit the Inspiring Communities website

Share this article:

Authentic Engagement is a Personal Journey

BY: NATASHA PEI

How do we engage authentically?

There’s really no systematic way of doing so. No bullet list will unlock the secret for an organization or collective to engage authentically without each of us being authentic individuals. After all, organizations are simply entities comprised of people – comprised of us.  

In our effort to professionalize the non-profit sector, we’ve forgotten how to interact with and care for people as people. We all want to be respected, heard, included, and made to feel like the contributions we make are important and valued. We all want to trust and be trusted.

I recently had the opportunity to attend a refugee info session for new sponsors. One of the big Aha’s! of the day was to treat people as people, not projects. Of course, as humans we’re not perfect. We stumble, sometimes get it wrong. But it is important that when we do, we pause, admit when we’ve gotten it wrong, and course-correct.

Easy to say in theory, more difficult in practice, right?

With so many policies, hierarchies and funding and deadline constraints, how do we show up with appreciative inquiry, rather than coming with a pre-determined agenda asking community members to buy into our plans and priorities? How do we fully invest in relationships?

Here is an example: Poverty Solutions Halifax. Their learning journey hasn’t been a perfect process. It’s been messy and complex. It is an example of engaging, listening, getting it wrong, re-learning, and re-acting.

Their Collective Impact, by virtue of the leadership team, was unreflective of the community as a whole. As a result, traditionally under-served communities again lacked established communication channels and trust to provide their insights throughout the development process. When the community came together to provide feedback, it was clear that certain communities, particularly African Nova Scotian, Indigenous communities, and rural residents weren’t adequately involved.

Despite the thoroughness of their engagement plan, particularly including all four sectors, the reality was that each community had a very different context with different intricacies. The key in their case was being truly open to feedback from the community and pivoting when they heard there were serious issues with the process. Adapting to what they heard was the first step in establishing trust and forming new partnerships.

They paused, reflected and re-strategized. The lead organization, United Way Halifax, internalized the lessons learned from the collaborative process and is using them to shift the way the people and the organization itself works, which will make a difference for all of their future projects as well. They are strengthening partnerships with the African Nova Scotian and Indigenous communities, are going on a journey of reconciliation and trust-building, and making intentional efforts to go out into the community to engage people where they are.

Importantly, United Way staff and volunteers are doing cultural humility training. They are learning about Indigenous history and current reality through the Mi’kmaw Native Friendship Centre, are participating in blanket exercises, having Kitchen Table Dialogues, watching a documentary series on cultural humility, planning visits to many community organizations such as the Black Cultural Centre for Nova Scotia, and more.

You can download the case study about their learning journey, here.

We can’t ask organizations to authentically engage without us, as people, walking that walk; just as we can’t ask organizations to journey on a path of reconciliation without us, as people, taking on our own personal reconciliation journey first or alongside our organizations.

Think about how you interact with valued family, friends and colleagues on a regular basis. What are the 5 key things that are important in those relationships? Now consider, does your organization’s or collaborative’s community engagement practices reflect these principles? If not, what can you do as a person, to re-humanize the process? 

Learn More:

• Read the Case study: Authentic Engagement – A Learning Journey
• Read refletions from Sylvia Cheuy on - Trust: An Essential Ingredient in Authentic Community Engagement
• Check out the paper The Context Experts by Lisa Attygalle
• Dive deep into Turf, Trust, Co-Creation and Collaboration by Liz Weaver

Share this article:

Principles Over Prescriptions

BY: GALEN MACLUSKY

Wouldn’t it be great if there was a recipe for community change? A simple, step-by-step process to follow? The problem is that our challenges are too complex and our communities are too unique to ever be encapsulated by a single prescriptive approach. Even robust frameworks like Collective Impact and the body of work surrounding it are not a substitute for grit, creativity, and flexible adaptation to the needs of our community.

In the absence of a prescription, what can help are principles that guide action – heuristics, values, and guiderails that help us decide what to do and how we should do it. We all use principles to navigate our own way through life:

Do unto others as you would have done unto you

Do one thing a day that scares you

Leave no trace

Turn the other cheek

The advantages of principles over prescriptions are many. They help inspire us to strive for better things rather than just ‘getting it done;’ they help us reflect and assess our actions; they enable us to adapt flexibly to each context as it comes. Prescriptions do none of these things, which means they limit our ability to have positive impacts on our community.

So, if you find yourself wishing for a prescription, a process, or a blueprint for effecting change in your community, perhaps instead what you need are the right principles to help you decide what to do next. If that feels right, check out some of the principles referenced below.

Learn more 

• Michael Quinn Patton has written extensively on the importance of principles in innovation, as well as what makes a good principle
• The Lived Experience Advisory Council (LEAC) has compiled seven guiding principles for leadership and inclusion of people with lived experience of homelessness
• With approaches such as Human-Centred Design and Design Thinking, our collective conversations often focus on process and practices rather than the more useful principles

Share this article:

What Makes Your Neighbourhood Work an Asset-Based Community Development Process?

BY: HEATHER KEAM

Asset-Based Community Development (ABCD) is not a new concept. About 40 years ago, John McKnight and Jody Kretzmann, took a four-year learning journey, visiting twenty cities and talking to 2,000 people in neighbourhoods which culminated in the book Building Communities From the Inside Out: A Path Towards Finding and Mobilizing a Neighborhood’s Assets. They wanted to discover if people knew who lived on their block and if they do things together. From this journey, 125,000 books have been sold and a movement was created. 

In the Spring of 2018, Tamarack hosted an ABCD workshop where we heard questions from participants about what makes ABCD different from other community development models. We found the answer in a recent article by John McKnight and Cormac Russell called Four Essential Elements of an Asset-Based Community Development Process. In this article, McKnight and Russell highlight four key elements to an asset-based community development process that other community development models do not require.

The four elements are:

1. Resources – There are six assets or resources which are used to enhance local wellbeing in every sense of the term:
   • Contribution of Residents
   • Associations
   • Local Institutions
   • Local Places
   • Exchange
   • Stories

2. Methods – this involves identifying and productively connecting unconnected local resources:
   • Starting with what residents can do themselves as an association of citizens, without any outside help
   • Next, looking at what they can do with a little outside help
   • Finally, once these local assets have been fully connected and mobilized, citizens decide collectively on what they want outside agents to do for them

3. Functions – involves seven functions that are critical features of all home-based natural communities.
   • Enabling Health
   • Assuring Security
   • Stewarding Ecology
   • Shaping Local Economies
   • Contributing to Local Food Production
   • Raising our Children
   • Co-Creating Care

4. Evaluations – Evaluating the extent to which citizens are engaged with the first three essential elements. This process of engagement is not about auditing, but about learning, and making midcourse corrections that allow us to stay committed to our cultural calling. The four evaluation principles are:
   • It identifies the maximization of gift exchange
   • It identifies the maximization and deepening of associational life
   • It attends to the maximization in the number of participating and co-producing residents and the increase in their citizen power. It places a particular emphasis on the inclusion of those who have been marginalized.
   • Sponsors of ABCD processes ensure that associated evaluations actively conform to the preceding three principles

The article concludes that for a community development process to be called Asset-Based Community Development it requires all four of the above elements at some point in the development process.  You can read the full article and learn more about the four elements to ABCD here.

ABCD was launched in Canada three years ago and Tamarack is working on building and supporting a movement for those who are working in community development and want an asset-based focus.  To continue this conversation about ABCD join our ABCD Community of Practice.

Learn more

• Read Four Essential Elements of an Asset-Based Community Development Process
• Visit the ABCD Canada website
• Join the ABCD Community of Practice

Share this article: