Big Aspirations: Constructing a Collaborative Evaluation Culture to Support the Impact Goals of an Intermediary Organization

Abstract

Intermediary organizations seek to make change in the world by strengthening the practice or building the capacity of direct service providers. In the education sector, these types of organizations often play important roles as technical assistance providers, capacity builders, additional administrators, investors, or funders. At the core of this capstone is a central question: how might intermediary organizations working at the institution-level define, articulate, and evaluate their own societal impact when most of the population-level changes they hope to see are so heavily dependent on actions taken by others? My research suggests that answering this question is very difficult, but I argue that building an organizational culture that tries to do so is both an ethical obligation and a pathway to cycles of continuous improvement, greater strategic focus, and staff satisfaction. I explored these ideas through my residency at Equity Meets Design (EMD), a for-profit capacity-building company whose clients cumulatively work directly or indirectly with approximately fifty percent of all public-school students in PK-12 and higher education institutions in the United States. This capstone chronicles my efforts to support EMD’s desire to use data more effectively as it works towards its mission to “increase the ability of organizations to (re)design interactions, interventions, and institutions towards increased equity in process and outcome.” While I do not arrive at a ready-made formula that offers quick and replicable answers, my analysis offers a roadmap for how intermediaries can 1) systematically embed evaluation practices into routine work, 2) practice using an equity lens to shift the definition, collection, and use of data, and 3) approach evaluation as an essential part of being in relationship with stakeholders.