By Nancy Muthee, Researcher, Inclusive Evaluation at Includovate
There has been a lack of open dialogue on evaluations and the varying power dynamics that surround Monitoring and Evaluations (M&E). With the increased recognition of M&E’s role in responsible governments, good governance, and evidence-based policies, there is a need for more open discussions on aspects that are often neglected yet are useful if we are to have inclusive evaluation approaches.
One potential area that can significantly ensure a standardised and context specific approach is through national guidelines. Most countries are developing National Evaluation Policies due to the increased recognition of the role of M&E. A global mapping of the status of National Evaluation Policies (NEPs) in 113 countries shows progress in NEPs from 2013 to 2021. What is interesting to see is the diversity of countries with NEPs, from less developed countries such as Azerbaijan and Djibouti to more developed countries such as Finland and Japan showing their recognition of the importance of evaluation.
NEPs are formal policies defining the purpose, responsibilities, functions, and organisation of public-sector evaluation in a particular country. In 2013, there were only 20 countries with a NEP; by 2021, that number had risen to 35 and there were 21 additional countries that do not have a NEP but conduct evaluations routinely. Often, where countries are routinely engaged in evaluation, they will eventually formalise the process through the NEP. For example, countries that did not have a NEP in 2013 and 2015 that routinely practiced evaluation initiated NEPs in 2021 (Rosenstein & Kalugampitiya, 2021). While having a NEP is a step in the right direction, the question of inclusive evaluation is not involved in the evaluation process itself in most cases. What really is inclusive evaluation? We define ‘inclusive evaluation’ as evaluation processes that are responsive to the needs of all groups including marginalised and vulnerable communities, allowing for results that are relevant for all groups concerned.
How do we advance inclusion within the evaluation field?
Monitoring and evaluation as a field is complex, skill-intensive, and has several facets. An enabling environment supported by policies, academic guidance, and common understanding across sectors is valuable.
The NEPs provide stakeholders with procedural rules and guidelines, and while that is welcomed, we cannot underestimate the importance of multi-stakeholder engagement in this field. A 2019 conference with the UNDP Independent Evaluation Office entitled, “Leaving No One Behind: Evaluation for 2030” agreed that building an evaluation culture was a “job for us all.” Participants agreed that “national evaluation capacities” mean more than the individual capacity of evaluators because they refer to institutional capacity and most NEPs have also identified partnerships as key to successful evaluations. How the different stakeholders and partnerships will evolve in the next few years will determine how the field moves forward. No one should be left behind particularly marginalised, vulnerable groups. Including different sectors and populations allows for every voice to be heard and for evaluation results to be objective without necessarily favouring a certain group. After all, evaluation is a governance and management tool that informs decisions that impact everyone.
A review of the United Nations system echoes the importance of capacity-building in the field and highlights that there is a need for knowledge and expertise exchanges to strengthen institutional capacities.
Voluntary Organisations for Professional Evaluation are non-profit membership organisations that are open to individuals interested in evaluation and are found in several countries under different names. These professional networks could play a critical role in these knowledge exchanges, and have an important role in guiding governments, private entities, and civil society in their countries on understanding evaluation and its role in evidence-based decision-making, organisational learning, and better informed public policy.
Asia-Pacific National Evaluation Policies and Systems
In an effort to encourage more dialogue, the Asia-Pacific region organised a dialogue forum on National Evaluation Policies and Systems on 13 December 2021. Speaking at the event, the Honourable Natalia Nikitenko, Member of Parliament of Kyrgyzstan, Chair of the Eurasian Parliamentarians Forum for Evaluation and Global Parliamentarians Forum for Evaluation, emphasised the need to engage decision-makers in the evaluation field to work with stakeholders, parliamentarians, government, donors, international development practitioners, academia, NGOs and media.
Presentations from Asia-Pacific countries during the dialogue forum show that much work had been done at the institutional level. At least 12 of the 14 Asia-Pacific countries that presented have a designated national public institution that leads monitoring and evaluation functions in-country. Institutional arrangements varied; in some countries, there is a specific ministry responsible for monitoring and evaluation, and in other countries that function is integrated across several ministries. For example, in India, the Ministry of Finance has made it mandatory for ministries to get central government-sponsored programs and schemes evaluated for budgetary allocation. In Nepal, evaluation is integrated into the country’s constitution, with a National M&E Bill drafted in 2019, while Japan and the Republic of Korea have bills/Act in place. Bangladesh, Japan, Nepal, New Zealand, the Philippines, and the Republic of Korea have evaluation guidelines, and Japan, New Zealand, and the Republic of Korea have evaluation standards as well.
What is interesting with such regional dialogues is that they allow for discussions from different perspectives. In this instance, the Asia-Pacific forum allowed knowledge exchange between nations representing the developing world with more developed nations providing both groups with an understanding of the local context and realities on the ground.
Despite this progress, institutional capacities remain relatively weak in most countries. While standards are important, we continue to see that, there is really no one size fits all as circumstances differ.
So what do we need to advance inclusion within evaluation? Many have highlighted that there is a need for active participation of stakeholders in the process. While we all realise the importance of engaging stakeholders, the process and outcomes of doing so may vary. Dialogues that engage and facilitate knowledge sharing between high and low income countries can reduce the existing power imbalance that has long surrounded the development and evaluation spaces. This marginalisation and exclusion has often led to programmes that do not respond to the needs of the communities.
About the Author
NancyMuthee is an Inclusive Evaluation Researcher with Includovate. She has over 4 years of experience in Monitoring, Evaluation and Learning (MEL) with an interest in social impact measuring. She has worked with organisations from the Global South and North and has a great understanding of the power dynamics existing in MEL having worked with donor and government agencies and grassroots organisations. She encourages the use of context-specific and inclusive methodologies.
Includovate is a feminist research incubator that “walks the talk”. Includovate is an Australian social enterprise consisting of a consulting firm and research incubator that designs solutions for gender equality and social inclusion. Its mission is to incubate transformative and inclusive solutions for measuring, studying, and changing discriminatory norms that lead to poverty, inequality, and injustice. To know more about us at Includovate, follow our social media: @includovate, LinkedIn, Facebook, Instagram.
