By: Shadia Nassar – Includovate/ Principal Researcher
In the previous blog, we delved into the causal analysis research method, which involves identifying cause-and-effect relationships between variables to understand the factors that lead to a specific outcome. Building on this concept, our focus in this blog will be on contribution analysis, a complementary research method that seeks to determine the extent to which different factors contribute to a particular outcome. Instead of establishing causal relationships, contribution analysis assigns importance or weight to various factors influencing the outcome. By utilising this method, we can determine the factors with the most significant impact on the outcome and explore how they interact with one another.
The contribution analysis term was coined at the end of the 1990s by John Mayne (2000) to provide an alternative to counterfactual thinking. It is a method used in research and evaluation to understand and assess the impact of a particular intervention, programme, or policy on a specific outcome. It involves identifying and analysing the contributions of different factors or variables to the observed changes in the outcome of interest.[1]
To build a contribution claim, it is important to provide clear and compelling evidence to support the assertion that the intervention has had a measurable and meaningful impact. Ultimately, the strength of a contribution claim lies in its ability to convincingly demonstrate the link between the intervention and the outcomes achieved. By carefully grounding the claim in a well-defined Theory of Change and supporting it with robust evidence, stakeholders can have confidence in the intervention’s effectiveness and its ability to make a positive contribution to the lives of those it serves.
The importance of contribution analysis in research and evaluation lies in its ability to provide a deeper understanding of the impact of interventions, programmes, or policies. By identifying the specific contributions of different factors to the observed changes in outcomes, researchers and evaluators can better assess the effectiveness and efficiency of interventions and make informed decisions about where to allocate resources for maximum impact.
Additionally, contribution analysis helps to improve the credibility and rigour of research and evaluation studies by making the process of causal inference more transparent and systematic. By explicitly specifying the contributions of different factors to the observed changes in outcomes, researchers and evaluators can more effectively address issues of internal validity and potential confounding variables.[2]
The Significance of Causality in Contribution Analysis
Causality is a fundamental concept in contribution analysis, as it helps to determine the relationship between different variables and their impact on the outcome of interest. Understanding the causal relationships between different factors can help identify the drivers of change and ultimately inform decision-making and resource allocation.
By tracing causal links between different factors and outcomes, contribution analysis helps entities understand the extent to which specific interventions or investments are responsible for changes in desired outcomes. This information can help stakeholders prioritise investments in programmes or initiatives that are most likely to have a positive impact on the goals and objectives of the project.
Causality is also crucial for evaluating the effectiveness of different strategies or interventions. By isolating the causal effects of specific factors, stakeholders can assess the impact of different interventions, identify areas for improvement, refine their strategies and make more informed decisions about resource allocation. [3]
Analysing Impact: Practical Examples of Contribution Analysis in Evaluation
Many development programmes contain overlapping activities that are frequently adjusted to changing circumstances and involve collaboration with various partners. Traditional evaluation methods that compare initials and results are not effective in these situations. Contribution analysis is a systematic approach that can better handle these complexities, involving a process of refining the theory of change and nested impact pathways. This is followed by using mixed methods research designs to confirm important assumptions. [4]
One of the evaluations used in the contribution analysis was a public health intervention aimed at increasing fruit and vegetable consumption in primary school children in NSW. The study tested the postulated theory of change against alternative explanations to examine mechanisms for change and factors influencing the implementation of the Crunch & Sip® programme. The analysis aimed to verify assumptions underlying the programme logic, assess the impact of external factors on programme outcomes, and evaluate the contribution of the programme towards increasing fruit and vegetable consumption in primary school children. The research focused on identifying factors that assisted and limited the implementation of the programme and determining the degree of influence of different mechanisms on achieving desired outcomes. In conclusion, this methodology helped reduce uncertainty by examining factors influencing outcomes and focusing on alternative explanations and contextual conditions that affect programmatic achievement. By analysing mechanisms for change, the study gained a deeper understanding of the programme’s contribution and the degree of influence of different factors on outcomes.
Another example of evaluations using contribution analysis is Using contribution analysis to evaluate health professions and health sciences programmes The study used Mayne’s six-step contribution analysis to evaluate the outcomes of six health professions programmes at an Australian university. It identified graduate outcomes and developed a theory of contributing factors. The study defined outcomes and identified cause and effect assumptions. It used existing data to build a model illustrating relationships between external influences, outcomes, results, assumptions, and risks. Six focus groups with academic staff, clinicians, employers, and graduates provided in-depth contextual information on factors supporting and hindering teaching and learning in health professions. These shared narratives helped refine the logic model for the final contribution story. Four themes were identified, describing learning and teaching strategies to enhance graduate outcomes. Only quotes from the focus groups were used in the presentation of findings, as participants added context to the gathered evidence.
Overall, these evaluations using contribution analysis highlight the importance of examining mechanisms for change, assessing external factors, and understanding the impact of different factors on programme outcomes. By using a structured approach, such as Mayne’s six-step contribution analysis, researchers can gain a deeper understanding of the contributions of programmes and interventions towards achieving desired outcomes. This methodology not only helps reduce uncertainty but also provides valuable insights into the factors that support or hinder programme implementation.
Conclusion
Contribution analysis can be a valuable tool in evaluating the impact of interventions or programmes, but it can also be challenging to implement in practice. It requires careful consideration of various factors and inputs, and it can sometimes be difficult to identify and quantify contributions.
However, I believe that while contribution analysis may not make the implementation of programmes any easier, it can provide valuable insights and guidance for decision-making. By systematically examining contributions and impacts, organisations can identify areas for improvement, allocate resources more effectively, and ultimately achieve greater success in their goals.
While the theoretical framework and methods of contribution analysis may enhance credibility with stakeholders, the actual implementation can be challenging. Gathering and analysing data, identifying causal relationships, and communicating findings in a compelling way all require time, expertise, and resources.
Closing Thoughts
Although contribution analysis is a method used to evaluate the impact of various factors on a particular outcome, it also presents several ethical complications in the context of the AI revolution. One major concern is the potential bias and discrimination inherent in the algorithms used in AI systems. If contribution analysis relies heavily on data produced by AI algorithms, there is a risk that the results may reflect and reinforce existing societal inequalities. Additionally, the lack of transparency and accountability in AI decision-making processes raises questions about the validity and reliability of the contributions identified through such analysis. Furthermore, the ethical implications of using AI to automate certain tasks traditionally performed by humans, including contribution analysis, need to be carefully considered to ensure that human values and moral principles are upheld in the face of technological advancement. Overall, navigating the ethical complexities of contribution analysis in the era of the AI revolution requires a thoughtful and conscientious approach to balancing efficiency with fairness and justice.
About the Author:
Shadia Nassar is a principal researcher at Includovate With over 25 years of experience in monitoring and evaluation, focusing on gender-related issues, she is a passionate person dedicated to promoting inclusive and gender-responsive policies and practices. Her extensive background includes designing and implementing comprehensive M&E frameworks for projects across education, health, economic empowerment, governance, and social protection. She prioritizes incorporating a gender lens into her work to address the specific needs of women and marginalized groups. She has provided technical support and training to project staff, government officials, civil society organisations, and community members through her collaborative approach to enhancing gender equality outcomes. A Ph.D. holder in educational psychology from the University of Jordan, she is committed to advancing her knowledge and skills to contribute to a more equitable and inclusive world.
References:
(1) Mayne, J. (2019a). A BRIEF ON CONTRIBUTION ANALYSIS: PRINCIPLES AND CONCEPTS. In United Nations Economic Commission for Africa (UNECA) (pp. 1–14). United Nations Economic Commission for Africa (UNECA). https://repository.uneca.org/bitstream/handle/10855/49332/Chapter%2018_%28resource%20doc%29%2014pp%2C%20BriefonCA.pdf?sequence=382&isAllowed=y
(2) Mayne, J. (2019b). Revisiting Contribution Analysis. Canadian Journal of Program Evaluation, 34(2). https://doi.org/10.3138/cjpe.68004
(3) Mayne, J. (2019a). A BRIEF ON CONTRIBUTION ANALYSIS: PRINCIPLES AND CONCEPTS. In United Nations Economic Commission for Africa (UNECA) (pp. 1–14). United Nations Economic Commission for Africa (UNECA). https://repository.uneca.org/bitstream/handle/10855/49332/Chapter%2018_%28resource%20doc%29%2014pp%2C%20BriefonCA.pdf?sequence=382&isAllowed=y
(4) Contribution analysis | Better Evaluation. (2021, November 2). www.betterevaluation.org. https://www.betterevaluation.org/methods-approaches/approaches/contribution-analysis