GEMINI MedED: Leveraging ‘Big Data’ to Understand Clinical Practice Variation in Resident Physicians

Abstract

Introduction: Residency training is an important source of clinical practice variation, given the known association between the training environment and future patient outcomes. However, current assessment approaches in postgraduate medical education do not routinely measure practice variation during training, leaving a gap in understanding how residency education could be optimized to improve patient care. Objectives: This retrospective cohort study aimed to measure and interpret clinical practice variation between senior internal medicine (IM) residents. First, quality improvement and clinical practice guidelines were used to select candidate resident-sensitive quality measures (RSQMs), defined as measures which were meaningful to patient care and mostly attributable to residents. Second, RSQMs were used to measure and interpret potential causes of practice variation. Methods: This study used electronic health record (EHR) data from the General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED), which links 793 senior IM residents at the University of Toronto to clinical data from 132,291 patients between 2010-2019. Ten RSQMs were selected related to pneumonia-specific and general IM care, then categorized based on the expected patterns of variation. Descriptive statistics were used to characterize variation in each RSQM. Results: Nine of ten candidate RSQMs were ultimately included. First line antibiotic ordering in pneumonia was performed for only 52.6% (7,085 / 13,470) of patients with wide variation. Wide variation in performance was observed for metrics categorized as discretionary (context-dependent). Potentially inappropriate transfusions were performed for 0.02% (26 / 132,291) of patients with low variation. Discussion: This study of 793 senior IM residents who cared for 132,291 newly admitted patients over 10 academic years demonstrated wide variation in clinical practice for pneumonia and general IM care. By increasing our understanding of how to characterize resident clinical practice variation, these findings offer several conceptual insights: (1) Considering the educational potential of identified variation, (2) A broader conceptualization of attribution as intentionally titratable and relevant to both diagnoses and patients, and (3) The need for a concept-driven approach to guide the use of EHR data. Overall, this study represents an incremental step towards better alignment of feedback, learning, and assessment in residency education with improved patient outcomes.

ERRATUM 2026-05-13 This manuscript characterized RSQM data using the mean, standard deviation, and interquartile range (IQR) as key descriptive statistics (Table 7). However, because most variables were skewed, the subsequently peer-reviewed published manuscript reported medians and IQRs instead. Additionally, after completion of this thesis manuscript, an error was identified in the initial query used to capture second-line antibiotic orders. Correcting this error resulted in the identification of additional second-line antibiotic orders. Although the conceptual findings and overall conclusions of this manuscript remain unchanged, readers are encouraged to consult the final peer-reviewed publication for the definitive quantitative results of this study: https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2848775