Person:

Woolcock, Michael

Beyond assessing whether or not interventions have achieved their stated goals, evaluations can also provide greater understanding—in real-time and at completion—about how, where, for whom, over what time frame, and which aspects of an intervention may (or may not) have succeeded, and thus where improvements might be sought. Moreover, to the extent any intervention is only as good as its implementation, evaluations can also help identify where breakdowns in the delivery system may have occurred and spaces wherein frontline implementers were able to find innovative solutions to local (“binding constraint”) problems. Process and implementation evaluations thus serve the vital purpose of jointly promoting accountability and learning, thereby expanding the common perception of evaluations as external instruments of compliance and discipline to internal procedures for promoting partnerships, innovation, and improvement (organizationally or technically). In this chapter, we describe six different types of process and implementation evaluations and describe their respective strengths and weaknesses in various contexts, for various purposes. As part of collective efforts to enhance the effectiveness of all classes of interventions, impact and process evaluations should be regarded as necessary complements.

Since March 2020, the novelty, intensity, and scale of the COVID-19 pandemic has placed enormous stress on governments, delivery systems, and social order around the world, especially so in countries with modest public health resources, where targeting is especially difficult, and among occupational groups working in close proximity to others. Those overseeing Indonesia’s health care system from mid-2020 onwards faced precisely this vortex of existential challenges, but certain organizations within this system were well placed to deploy an adaptive implementation strategy. KOMPAK was one such organization2; its efforts were not universally successful, but their achievements (e.g., coherently coordinating governance efforts between national, regional, and local levels of government; building effective village information systems) were nonetheless distinctive, consequential and enduring. In this sense, COVID-19 can be understood as a perverse but instructive “natural experiment” in how well public sector organizations respond in the face of unexpected high-stakes, high-uncertainty, low-resource, low-prior-experience crises. We document key insights and implications for public sector administration from KOMPAK’s efforts, not just for Indonesia and other developing countries but for public service delivery systems more generally.