A Conceptual Model for the Evaluation of Surgical Missions

Abstract

Background: Medical missions to low and middle-income countries are increasingly frequent, with an estimated 6,000 trips accounting for ~ 200,000 surgical cases and over $250 million in costs annually. However, these missions have received little critical evaluation, which has prompted questions about their sustainability, cost-effectiveness, and quality of care. In response to these critiques, a few proposed frameworks for the evaluation of missions have been published. While these publications offer theoretical frameworks that detail topics for evaluation, they do not agree on which aspects should be assessed and they have not been shown to quantify or change the practices of an established mission. This paper presents proposes an evaluation model for surgical missions based on the research program of Operation Walk (Op-Walk) Boston, a mission trip to the Dominican Republic that provides total joint arthroplasty. Methods: We began the construction of the evaluation model by using the Center for Disease Control and Prevention’s (CDC) “Framework for Program Evaluation in Public Health” to analyze Op-Walk Boston’s prior research efforts. This led to the creation of a logic model to better visualize Op-Walk Boston’s work stream for providing care. Additionally, the logic model delineated the different aspects of the mission that should be evaluated. In order to compare how our evaluation categories compared to other evaluation models, we conducted a literature search for review papers on medical missions as well as prior evaluation models. Using the information gathered from the CDC framework and the literature, we propose an evaluation model that calls for an assessment of the salient contextual factors (e.g. culture & beliefs), system management (structure, process and outcomes), and the sustainability of the program interventions. We then used these domains to present findings from the quantitative and qualitative research work of Op-Walk Boston. Results: Op-Walk Boston’s qualitative research findings demonstrated that cultural factors are important determinants of patients’ perceptions of arthritis etiology, physical activity patterns and treatment preferences. Quantitative assessments documented that the Dominican patients had worse lower extremity functional status (mean WOMAC function score of 33.6 compared to 43.3-54) and pain preoperatively than patients undergoing total hip or knee replacement in the US, yet they achieved excellent outcomes (fifty-point improvement), comparable to those of their US counterparts. Assessments of the quality and of sustainability of the program showed that the quality of care provided by Op-Walk Boston meets Blue Cross/Blue Shield Center of Excellence criteria and that sustainable changes were transferred to the host hospital. Finally, we assessed the effectiveness of Op-Walk Boston’s prior research work using the CDC’s recommended standards for effective evaluation implementation and determined that it met appropriate levels of utility, feasibility, propriety, and accuracy. Conclusion: Our proposed model offers a method for the formal assessment of medical missions that addresses the call for evidence of their merit. The prior research efforts of Op-Walk Boston within each of our suggested evaluation domains have improved the work of the mission by detecting both the strengths and inadequacies of the program. We suggest that surgical missions adopt quantitative and qualitative strategies to document their impact, identify areas of improvement and justify program continuation, growth and support.