Inclusive rigour

Inclusive Rigour is a way to reframe “rigour” in order to produce more useful and meaningful understanding of how change happens based on epistemological pluralism.  It was developed to move away from a restrictive understanding associated with a strict, method fidelity-driven rigour which in evaluation is often associated with counterfactual designs  towards a more holistic understanding based on a thoughtful design process, that takes into account complexity and non-linear causal pathways.

Inclusive Rigour posits that a wide range of participant experiences are key in understanding change in complex systems , because they have agency in the process of change, have unique experiences of the system and what supports or hinders change that are necessary to explain how causal pathways are emerging. .. When participation is power aware and meaningful, it can lead to co-ownership of the evaluation process, supporting a more transformative approach to evaluation.

The concept was first introduced in the field of evaluation by Robert Chambers (2015), and further developed by authors such as Preskill & Lynn (2016), or Aston & Apgar (2022).

The Inclusive Rigour Co-Lab is a diverse group of evaluators, practitioners, researchers and funders reflecting on the practice of inclusive rigour when evaluating in conditions of complexity. To enhance practical applications of this conceptual reframing as well as to support debate and learning , the Inclusive Rigour Co-Lab used  case studies across their broad range of complexity-aware peacebuilding initiatives to shape a framework, published in this 2024 article. 

Other similar communities of practice are also reconceptualising rigour to broaden discussions about quality and rigour in evaluation in similar ways (such as the Causal Pathways Initiative for example).

Design and principles

The framework published by the Inclusive Rigour Co-Lab is composed of three main domains of practice, which all should be taken into consideration during an evaluation design process. These elements interact, and they also all exist within a wider context.

• Utilisation and impact is where we strive to negotiate different stakeholder (including participants, programmers and funders) needs for evidence and learning to inform decision-making and achieve the ultimate goal of increasing the impact of peacebuilding programmes on the ground.
• Facilitating meaningful participation and inclusivity is how we pay attention to the ways in which our processes engage with power and open up or close down space for different forms of knowledge, particularly of the most marginalised, to be included meaningfully.
• Achieving effective methodological bricolage includes  making decisions about appropriate design, methodological recombination and choices of specific methods and tools we bring together for the purpose of understanding and evaluating causal pathways.

The wider context in which these three domains of practice intersect and are enmeshed in is the Enabling environment, which encompasses both institutional dynamics and personal and team competencies.

Institutionally, the command-and-control management practices and associated cultures they are part of continue to be perpetuated daily. Shifting these cultures towards learning is at the heart of enabling effective methodological bricolage. This includes managing uncertainty and moving away from methodological fidelity to create space, time and budget for flexibility and iterative co-design.

Individually, the competencies required to make this shift have been described within the context of participatory evaluation and include: sound facilitation skills and reflexivity; humility and honesty; balancing principles with pragmatism and understanding the political landscape. These individual competencies are enabled and supported through team and institutional competencies.

The full aims and composition of the framework are explained in this video in English, French and Spanish.

Further practical guidance on decision-making for evaluation and learning design is being developed. Both the framework and its associated guidance are designed to be iteratively updated, rather than fixed in time. Updates will come as a result of practical use supporting specific conceptual or practical refinements.

Mini case study

This mini case study aims to showcase the value of using the conceptualisation of inclusive rigour to critically reflect on quality in a complexity-aware MEL design and process.

We use, the ‘Vestibule of Peace’ project, which was implemented between 2019 – 2024 and responded to the critical failure to meaningfully include local communities in the Malian peace process, as it was one of the original inclusive rigour cases that IDS was involved in.

It was co-designed and implemented by a Consortium of four partner organisations: (1) L’Institut Malien de Recherche Action pour la Paix (IMRAP),  (2) Interpeace, (3) IDS, who played a methodological accompaniment role and (3) Humanity United (HU) whose peacebuilding strategy centred on building local agency as a vehicle to transform the peacebuilding system. The collaboration used Systemic Action Research (SAR) as an alternative to the conventional externally driven conflict management mechanisms which had revealed their limits through the 2012 crisis in Mali.

As is described by Apgar and colleagues (2024) the programme’s MEL system responded to the opportunity of developing and testing SAR as a dynamic and evolving methodology to locally owned and driven peace. It was co-designed by all Consortium partners with two objectives: (i) to facilitate actionable learning as the SAR process was implemented in order to fuel adaptive management; and (ii) to generate credible evidence and respond to agreed evaluation and learning questions. The evaluation design was based on a combination of contribution analysis with causal theories of change which were then explored through participatory process evaluation (via case studies), and outcome harvesting for causal analysis of emergent pathways.

The overarching evaluation question for the programme defined by the theory of change was: How does the SAR process at community level contribute to building the conditions for community driven peacebuilding?  Within this, we focused on learning about the action research groups, asking what outcomes emerge directly through the action research group process for whom and under what conditions? We also were interested in understanding what broader processes of change result from the SAR process in local conflict resolution?

Using the inclusive rigour framework we here critically reflect on if and how the bricolage design supported meaningful participation. Through the SAR intervention itself, participants were involved in developing their own actions and evaluating their effectiveness. This grounding of the MEL design in the participatory intervention supported deepening participation, including use of local harvested for collection of outcome descriptions in the participatory outcome harvesting process. However, analysis remained in the control of the programme implementation team, missing an opportunity to support greater ownership and use of the evaluation findings by local communities. The design was intentionally aiming to support adaptive management, and emergent learning was used by the implementation team throughout. It was not, however, till the final evaluation was completed, and analysis that reflects back on the theory of change, that use beyond the internal programme needs becomes possible.

The main learning from this analysis was around the tensions held by different partners on the purpose of evaluation which created at times a disabling environment for the participatory evaluation. Some partners held on to views of evaluation as serving performance monitoring and struggled with a MEL design that was not business-as-usual.