The contribution of Monitoring and Evaluation to effective policy making through useful data

Quality data has often been overlooked by decision makers as something trivial, too tedious and time consuming for their necessary attention. Often, the notion of data and its usefulness is undermined, resulting in an ad hoc inefficient use of resources as well as poorly implemented social development interventions and policies. Albeit, often data that is presented is not simple to understand and is presented in a complex manner contributing to the lack in enthusiasm in interpreting and using it. This has given rise to the increase usage of infographics which depict accurate, simplified data that is easy to remember, yet due to the jadedness of data and its meaning, importance and relevance, insufficient attention has been given to mandate the collection of useful data.

Decision-makers across the world need to base their decisions on information from reliable sources. This is imperative to evidenced based decision making through which relevant, implementable and impactful policies and developmental interventions can arise. They need to learn from the best evidence based knowledge and experience available and they need to know what kinds of research and what type of data could help them make the right choices. Successfully implemented and impactful polices and developmental interventions hinge on appropriate and well-designed monitoring and evaluation frameworks which depend heavily on useful quality data. However, practical work in the development field across sectors has often demonstrated that either too much data is collected with little regard for the quality, or that data is missing, or that useless data is collected because monitoring and evaluation frameworks that clearly articulate what type of data is necessary and relevant is missing.

Why is data missing and the importance of collecting relevant data?

The missing link between research, practice and policy is data that is accurate, valid and reliable. In Africa, there is a large gap between the producers and consumers of knowledge, and research could have a greater impact on development policy than it has had to date. Researchers as “knowledge makers” struggle to understand the resistance to policy change despite clear and convincing evidence whilst policymakers as “knowledge consumers” lament the inability of many researchers to make their findings accessible and digestible in time for policy decisions (Jones, 2011: 7).”[1] Furthermore, the politics surrounding access to information and permission to collect useful data negatively impacts the generation of useful research and subsequently current data. In other instances, slow bureaucratic processes and political bickering can further negatively impact the publication of data contributing to the absence of data to the general public. Additionally, fieldwork experience has highlighted the ignorance of monitoring and evaluation officers in understanding the importance of collecting, collating, analysing, interpreting and presenting user friendly data. Not only are records unkempt, but the nonchalant attitudes towards the value of data is disturbing.

Collecting relevant data is of paramount importance to sourcing the correct information regarding a socio-economic challenge from the beneficiary point of view. Developmental interventions are still heavily carried out with a top down approach with the target beneficiaries often complaining that what is given is not what is required to make the necessary changes the policy seeks to address. Before or during the policy design phase, it would make sense to carry out a situational analysis to understand the key target groups needs and priorities, the demographic factors and the capacity challenges of government, NGOs (non-governmental organisations) and CBOs (community based organisations) staff. There is sufficient evidence from my working experience in the development field that capacity is severely lacking in implementing the required policies hence, the collection of timely, useful and relevant data is frequently compromised. The diagram below illustrates an evidenced based policy pathway.[2]

                        blogpost

blll44Source: Adapted from Bowen and Zwi, (2005)

The above pathway to evidence-based policy and practice involves five phases which are:

  • sourcing the evidence,
  • using the evidence,
  • implementing the evidence,
  • identifying implementation gaps and
  • monitoring and evaluation

The policymaking context is exceedingly political and rapidly changing and depends on a variety of factors, inputs, and relationships. It is of paramount importance to ensure that monitoring and evaluation is part of the agenda setting of the policy to guarantee that useful, relevant and current data is collected. This will enable the assessment of the overall impact of the policy on the beneficiaries. Problem identification of the policy setting phase needs to be informed through research and a situational analysis- the situational analysis can enrich or negate the data of existing research and can necessitate the need of a national survey to update old or missing data. This process can further highlight the correct target group for the policy intervention and areas that are an urgent priority. Often a policy is drafted and only at the evaluation stage does it come to light that there is no useful data that can tell us what impact the policy had. It is only at this late stage that decision makers realise the need for accurate data.

Once the evidence is used to draft a policy discussion paper to make a case for the policy, there needs to be a discussion around what the data says and to ensure that the implementing partners of the policy have the capacity to implement the policy. In developing countries, often it is discovered that great polices are drafted however, there is limited capacity to implement it. Thus, in some countries, more structures are created wasting more financial resources rather than addressing the capacity constraints. From my work in the development field, a significant capacity constraint is work ethic coupled with the ignorance in understanding the job description of monitoring and evaluation officers when it comes to data collection, collation, analysis and presentation. In addition, the absence of understanding the importance of data by key management staff results in data collected in an ad hoc manner with no one being held accountable for the failure of missing data.

Once the policy is in place and is being implemented, monitoring needs to occur to collect data for ongoing activities and outputs and to inform management when the implementation diverges from the objectives of the policy. Monitoring further supports management when implementation of the policy itself has become futile enabling sufficient response time to address these issues bringing implementation back on track.  An evaluation has to be conducted to assess what impact the policy had on the target beneficiaries and what areas worked well and what did not and what could be done to improve the policy design.

Important to note is that neither policy nor monitoring and evaluation can occur without useful quality data. It is data that presents a case for a policy or development intervention and it is data that describes how, when and why the policy or intervention worked or did not work and provides insights as to how to improve for the future. The value of data cannot be underestimated and the mandate to ensure accountability for quality and timely collection of data needs to be championed by managers. After all, without data we do not know what the reality is on the ground is and how to address the issues that will result in impactful and meaningful change in the lives of the beneficiaries of the policy or intervention.

[1] Jones, B. 2011. Linking Research to Policy: The African Development Bank as Knowledge Broker, Series N° 131, African Development Bank, Tunis, Tunisia.

[2] Bowen, S., & Zwi, A.B. 2005. Pathways to “evidence-informed” policy and practice: A framework for action. PLoS Med, 2(7): e166.

Advertisements

Good Governance and Social Protection

Introduction

Governance and social protection are key emerging issues necessary to ensure the realisation of the forth coming sustainable development agenda. While not all the millennium development targets were achieved in any country, a number of positive strides have led to an improvement in some of the MDGs country outcomes. The MDGs and most policy objectives are manifested through development interventions. Often the impact of many interventions is unknown as determining impact is costly and requires time. When interventions fail no substantial explanation of why they failed are known and often broad based reasons are cited such as lack of capacity to implement. In the case of a lack of capacity, it is important to identify what about capacity is lacking? Is it data collection skills, is it data capturing skills or is it data analysis? Social protection is the right of every human being therefore it is important to know what the efficiency, effectiveness and impact of social protection programmes are on the lives of the beneficiaries especially in countries that have been upgraded to Middle Income Status but exhibit high levels of poverty and inequality. This dualism brings with it, its own challenges. Effective targeting becomes important. This therefore leads to the crux of this essay; what does good governance mean for social protection?

Governance versus Government: Are they the same?

To understand what governance means, it is critical to first understand that governance is distinct from government. There is a vast amount of literature in the field of public administration which draws and maintains distinctions between governance and government which together refer to the various facets of public problem solving and systems for managing society as a collective organism (Newland, 2002).

Government has now come to play the leading role in executing state distribution of public goods incorporating both old-fashioned leadership perspectives, historic public administration functions with a new management style. Government has come to be viewed as an organisational unit increasingly regarded as the public problem solver when it comes especially to service delivery of basic goods and services.

Government is motivated by the collection and distribution of goods through direct or indirect methods. It is done through a hierarchal means which is heavily bureaucratised and rigid with a focus on the local and regional areas (Reddel, 2002:59). There are rules and procedures to be followed in the distribution of these goods and may outsource services and provision of goods to improve efficiency in administrative functioning (Reddel, 2002:59).

Governance then is broader and refers to the dynamic, interactive and symbiotic relationship between stakeholders who may be public, private, semi-public and religious and even international bodies (Kamarack, 2002:8). It involves partnerships formed between these stakeholders rooted in a collective action approach in resolving complex and messy public problems. It makes room for the use of new tools and embraces a global perspective. This concept of governance echoes the beliefs of the German Philosopher, Habermas who advocated for both a strong state and a strong civil society as well as increased access to decision making, informed public decisions, novel forms of participation outside the strict confines of institutions and greater freedom for citizens (Reddel, 2002:57).

Governance discards the traditional model of public management theory where systems are centrally controlled and organised in hierarchies. It looks to the establishment of networks to achieve the optimal collective outcome (Mayntz,2003). New public management and new governance demands have emphasised the need for new management skills in deciphering contemporary public problem solving.

Hence governance can be summarised as collective actions taken through a collaborative effort in a decentralized manner through public, private or multi-level transnational agents at either a regional or global level in order to distribute a good.

Public accountability in the field of governance has been a major area of concern with constant innovation in improving methods, techniques and delivery of services and goods (Haque, 2000:599). Accountability in the context of public administration refers to the lawful and hierarchical views of responsibility as well as the satisfactory use and discretion of power vested in the authorities. Accountability then raises the questions of who punishes when the individual’s management of himself conflicts with that of the state and the general rules and procedures. The public has the right to know to whom ultimately is the state and bureaucracy accountable to especially in one party democracies. Contemporary changes in governance has led to more public accountability in terms of its objectives, constructions and roles towards efficiency, outcome, customer orientation and value for money (Haque, 2000:599).

Good governance therefore becomes the mode through which government and partners operate therefore are subject to accountability by the citizens. But more specifically, what does good governance mean for social protection? How does a citizen hold anyone accountable when the lines between governance roles, responsibilities, duties and outcomes are blurred? The next section explains what social protection is and the implications of good governance thereof.

Social Protection: More than Accountability

Social protection has become a growing focus area in development policy for a number of reasons namely, increasing inequality, globalisation and the impacts of the financial crisis as well as the demographic transitions of many countries. Part of social protection’s increasing relevance stems from the realisation that it can serve as a poverty reduction tool helping people shift away from dependency to some form of productive livelihood (Devereux and Sabates-Wheeler, 2007:1). Social protection seeks to combat poverty and inequality in an effort to tackle the multiple dimensions of poverty and exclusion. Rights to elements of social protection are ensconced in the Universal Declaration of Human Rights and therefore governments have a duty towards rights bearers in the delivering of these rights. Therefore good governance becomes critical in the ultimate realisation of these rights by citizens.

Good governance is a key pillar to improving service delivery and outcomes of policy induced interventions. Governance is more than an anti-corruption campaign, it entails improving incentives for policy makers and provides and strengthens accountability of services to citizens (Basset, Gianozzi, Pop and Ringold, 2012:1). Secondly, governance has come forth as a critical tool in achieving the sustainable development goals. Lastly, governance is increasingly recognised as important in achieving development outcomes through instilling evidence based research, policies, and monitoring and evaluation of development interventions.

There are a number of challenges encountered in the practical application of good governance. Firstly, what necessitates good governance and keeps it good, is political will and commitment. Without the support of the ruling elite, very little can be done in terms of creating, implementing and influencing policy. Secondly, social protection programmes need to be clear and concise with respect to their roles, objectives and outcomes. Overlapping unclear programmes make it incredibly difficult to assess the impact of any one programme on the beneficiaries. In some instances, one beneficiary may even benefit from multiple programmes adding on to inefficiencies and creating an environment of dependency. Thirdly, the absence of implementation guidelines adds on to programme inefficiency as each district may freestyle implementation leading to incomparable results and varying outcomes at the local level. This makes analysis messy and very little can inform decision making on what has worked and what has not. Lastly, governance requires structures that maintain and enhance accountability of both government and development partners. However, an often missing-in-action partner is civil society and a loosely connected weak civil society is unable to hold government accountable when service delivery fails. These institutions need to remain strengthened, independent and devoted to the implementation of development interventions in a politically impartial manner. Furthermore, it is imperative that government, development partners and civil society organisations understand that there are various types of schemes and interventions within the social protection domain that have different target groups, varying outputs, outcomes and impacts on the end line beneficiaries. Therefore their understanding of this has implications for how good governance is practiced.

Some recommendations that this paper presents for good governance in social protection are as follows:

  1. Social protection programme or project guidelines or frameworks need to be developed and should inform on how a programme is to be implemented; what the roles and responsibilities of each organisation, institutions and person is and develop standardised reporting formats and data collection instruments while establishing accountability mechanisms that are transparent and effective.
  2. Ensure that regular reporting, monitoring and evaluation of social protection occurs with the results being disseminated nationally and online so that reports and data are available for analysis by the public.
  3. Development of a database of civil society organisations that are involved in social protection programmes to enhance communication and attendance and establish a platform where practical knowledge sharing at national stakeholder meetings can occur.
  4. Mechanisms for shock such as natural disasters should form part of the public policy for social protection programmes instead of assistance for merely coping afterwards.
  5. Poor implementation as a result of poor capacity should be followed up with a bottleneck analysis, or an implementation evaluation to identify and understand what exactly hinders effective and efficient implementation.
  6. If implementation is stated as a problem due to a lack of capacity, then capacity building needs to be made a priority by the government.
  7. A coherent and systematic monitoring and evaluation framework needs to be developed with clear objectives, outputs and outcomes including a concise theory of change and log frame to guide management on progress of the programme.
  8. It is crucial that impact evaluations of projects and programmes occur to demonstrate whether a particular intervention made the intended impact and if not, to identify what the unintended impacts were and why the occurred.

Conclusion

Governance has become increasingly important in realising effective and efficient development interventions in terms of achieving the intended objectives, outcomes and impact. Understanding the various types of social protection programmes is critical as they have different activities, outputs, outcomes and impacts on the beneficiaries and therefore has implications for how governance is practiced. There are a number of challenges in maintaining good governance namely, lack of political will and commitment, poor or over lapping implementation of programmes, lack of implementation guidelines and poor accountability at the level of government and partners. Some key recommendations include i) development of monitoring and evaluation framework and programme implementation guidelines, ii) regular reporting, monitoring and evaluation, capacity building to improve programme implementation and iv) clear and concise accountability structures and mechanisms.

References

Basset, L., Gianozzi, S., Pop., L and Ringold, D. 2012. Rules, Roles and Controls. Governance in Social Protection with an application to Social Assistance. World Bank.

Devereux, S. and Sabates-Wheeler, R. 2007. Editorial Introduction: Debating Social Protection. Available from: http://www.ids.ac.uk/files/dmfile/IntroDevereux38.3.pdf. (Accessed 26 September 2015).

Haque, S.M. 2000. Significance of Accountability under the new approach to Public Governance. International Review of Administrative Sciences. 66: 599-617.

Kamarack, E.L. 2002. Applying 21st Century Government to the Challenges of Homeland Security. Available at: http://www.hks.harvard.edu/mrcbg/research/e.kamarck_pricewaterhouse_applying.21st.century.government.pdf. (Accessed 26 September 2015).

Mayntz, R. 2003. From government to governance: political steering in modern societies. Presented at the Summer Academy on IPP at Wuerzburg, September 7-11 2003. Available from: http://www.ioew.de/fileadmin/user_upload/DOKUMENTE/Veranstaltungen/2003/CVMayntz.pdf. (Accessed 26 September 2015).

Newland, C. (2002). Building the Future of Local Government Politics and Administration. In H.F.J. Nalbadien (Ed.), The Future of Local Government. Washington, DC: International City/County Management Association, pp. 231-245.

Reddel, T. (2002). Beyond Participation, Hierarchies, Management& Markets: New Governance &  Place Policies. Australian Journal of Public Administration. 61: 50-63.

WELCOME TO RADAF!

Dear Visitor

Welcome to RADAF (Research and Development Africa)!

We are a social development research and management consultancy focusing on issues pertaining to youth such as education, sexual health and reproduction, employment/unemployment, sustainable livelihoods and entrepreneurship. This blog is a platform for you to get involved through insightful experiences, relevant issues and challenges that you have faced or think are important with respect to the above focus areas. We believe in evidence based research through which intellectual debates can occur.

We hope you enjoy our blog and look forward to your posts!