What’s so stimulating about the language of evaluation?
On several recent occasions in meetings about designing evaluation plans for community interventions the terminology about evaluation has almost derailed constructive, creative and innovative conversations about an intervention, and all I keep thinking is “WOW! I am finding this really overwhelming, I can’t imagine how people without a background in evaluation must be feeling when having to make decisions about these things!?”
Over the past few months I’ve been settling into my position here at the Tamarack Learning Centre, and I've had the chance to take a long into the evaluation field and how people are talking about evaluation. I’m getting a sense of how the field use language - words like impact, metrics or indicators, measurement, or the big one itself evaluation. These words carry different meanings depending on your circles, context, community. Some of these words carry significant historical meaning that can block or negatively affect how we may understand the effects of interventions in communities, while these words can bring a great deal of clarity and hopefulness to others.
So what do we do?
Even among the sea of different perceptions, perspectives and experiences, one thing that becomes clearer is that the language of evaluation is capable of stimulating insightful, reflective, critical conversations that can eventually influence change.
Whether this change happens at a greater scale, such as at the community level or regional level, change happening at the individual level is a driving force. If we keep in mind that all systems are made of up individual people, then we begin to understand that these evaluation plans and systems approaches are decided upon by people carrying biases, attitudes, beliefs and knowledge. We ultimately need to find ways of building common ground, respect, and capacity to work together if the goal is to seek changes within an institutions, community, and beyond.
In the book Collaborative Approaches to Evaluation: Principles in Use, J. Bradley Cousins explains that when doing evaluation we should focus on the description of practices that meet the basic requirements of evaluators working hand-in-hand with non-evaluators to produce evaluative knowledge (2019: p.6). From my perspective, this notion starts with something as basic, but as powerful as language.
In order for us to do this work, below are three strategies that I hope are helpful for preparing a playing field:
1. Checking in with those you're working with to see if there is a common understanding of the evaluation terminology being used. Often I find myself in situations where the term impact is being used but really people are talking about an immediate outcome of their work. Even if we think we know what these terms mean, it’s helpful to check in with those at the table to ensure you’re all on the same page about your specific intervention these terms are being applied to. Doing this type of check-in can be as simple as asking some basic questions, such as:
- What does evaluation mean to you?
- How do you understand the term impact?
- Do we all understand what indicators or metrics are?
- Who’s missing? Are all the ‘right’ stakeholders weighing in on this conversation about evaluation language? Determining who the ‘right’ stakeholders to include in this dialogue can come from a process of identifying who they are. (see below for the Stakeholder Engagement Wheel process)
- Can we come to an agreement about how we define these concepts? If not, how do we want to handle this?
- How will the evaluation language we use affect the success of planning a robust evaluation process, and who participates?
3. Knowing when to engage the services of a professional evaluator to support this process of reflection and when to do it on your own. There are a variety of approaches to doing evaluation work out there. It can be a daunting task to decide on which one(s) to employ. As an initial first step, reflecting on and answering these questions might prove most useful to determining how and when to engage a professional evaluator:
- What is it you’re trying to achieve with the evaluation process?
- What is your existing capacity to design and conduct evaluation?
- Who is best situated to participate in the process?
- How might participation happen, and to what extent and when will participation of the various stakeholders and beneficiaries be involved?
- What is our evaluation vocabulary and how do we define it in the context of our work?
The extent to which evaluation language is understood can be most helpful. You might decide that engaging an evaluation professional to provide support on answering those questions above and defining the evaluation concepts and terms with you is also a nice option. It can certainly be a great way to begin building that evaluator relationship if you decide to work with them later on.
Take Your Learning Further:
Below are some great resources that can build and strengthen evaluation knowledge and perhaps be useful in knowing what questions to ask a professional evaluator when the right moment arises.
- Cousins, J. B. (Ed.). (2020). Collaborative Approaches to Evaluation: Principles in Use. SAGE Publications.
- Fetterman, D. M., Rodríguez-Campos, L., & Zukoski, A. P. (2018). Collaborative, Participatory, and Empowerment Evaluation: Stakeholder Involvement Approaches. The Guilford Press.
- TOOL | Stakeholder Engagement Wheel
- Evaluation Terminology Tip Sheet.
