System Feasibility, Usability, and Acceptability of an EHR-Based Patient Outcome Feedback System for Pediatric Intensive Care Unit Referrals

Abstract

Outcome feedback — the structured communication of patient outcomes from receiving physicians to referring physicians — has the potential to improve diagnostic calibration and clinical learning, yet scalable mechanisms for delivering feedback remain underdeveloped in pediatric acute care. This thesis describes the development and iterative evaluation of an electronic health record (EHR)-based system for delivering patient outcome feedback letters from the Medical Intensive Care Unit (MICU) and Pediatric Intermediate Care Unit (PIMCU) to referring Emergency Department (ED) physicians at Boston Children's Hospital (BCH). The system was developed and implemented across three iterative Plan-Do-Study-Act (PDSA) cycles from November 2025 through March 2026. Using a human factors engineering (HFE) framework, three outcomes were assessed: (1) feasibility, assessed through delivery pipeline tracking including workflow latency, routing accuracy, and qualitative interviews from unit physicians; (2) system usability, measured by the System Usability Scale (SUS); and (3) acceptability, perceived task load among unit (MICU and PIMCU) physicians, measured by the NASA Task Load Index (NASA-TLX), with qualitative debrief interviews from unit physicians alongside surveys from ED physicians providing additional context. A total of 25 patients were tracked; 10 unit physicians participated as feedback letter senders and 6 ED physicians as recipients. The overall mean SUS score was 84.3, surpassing the “acceptable” threshold of 68, and approaching the “excellent” usability threshold of 85. The overall mean NASA-TLX unweighted raw score was 14.1, indicating low perceived task load. Routing accuracy was 100% across all eligible patients. Qualitative analysis via debriefs identified five themes among unit physicians: (1) the EHR-based feedback system was universally acceptable; (2) the feedback workflow was low-burden; (3) although the workload 4 burden of reviewing, editing, and sending feedback letters was acceptable at low patient volumes, concerns about sustainability at higher volumes and for more complex patients were raised; (4) EHR-generated patient information was perceived as accurate and useful for only straightforward patient cases; (5) unit and ED physicians engaged constructively with iterative design process, generating actionable recommendations that directly informed improvements across cycles. ED physician engagement was limited by their tendency to rarely access the EHR inbox compared to other tabs in their daily workflow. Thematic analysis of ED physician feedback collected through surveys showed that they believed the system was valuable for learning from mistakes but would have greatest value in community ED settings where no existing feedback mechanism currently exists. These findings support the acceptability, feasibility, and usability of the EHR-based feedback system as a foundation for broader implementation and subsequent expansion for feedback delivery to external referring community ED physicians.