Person:

Saghafian, Soroush

Addressing hospital emergency department (ED) overcrowding is a critical challenge for many healthcare systems worldwide. Many hospitals (including our partner hospital) have been experimenting with innovative patient flow designs to address this challenge. A promising new design is to separate patients who can be served vertically (e.g., on a regular chair as opposed to horizontally on an ED bed) and route them to a different area termed the Vertical Processing Pathway (VPP) unit. While this can potentially increase operational efficiency by removing the burden caused by a main ED bottleneck—lack of bed availability—it can degrade performance if patients that are routed to the VPP unit need to be sent back to be served in an ED bed, or if some patients that could have been served in the VPP unit end up occupying an ED bed. Successful implementation of this design, thus, significantly depends on understanding which patients should be routed to the VPP unit and when. To assist our partner hospital and other EDs, we develop a machine learning model trained on large-scale data capable of providing a personalized risk score for each arriving patient on whether or not they will eventually need an ED bed. We then feed these risk scores to an analytical model of patient flow to characterize the optimal protocol for utilizing the VPP unit. We find that the optimal protocol depends not only on the predicted risk scores but also on the machine learning model’s accuracy as well as some of the main ED characteristics (e.g., patient arrival intensity and congestion level). To gain deeper insights, we make use of simulation analyses calibrated with hospital data and compare the performance of our recommended VPP-based patient streaming design with more traditional ED flow approaches such as “fast track” or “physician in triage.” Our results suggest that following the VPP design under our recommended protocol can bring several advantages to EDs, allowing them to significantly improve their operations.

There is a growing amount of evidence that machine learning (ML) algorithms can be used to develop accurate clinical risk scores for a wide range of medical conditions. However, the degree to which such algorithms can affect clinical decision-making is not well understood. Our work attempts to address this problem, investigating the effect of algorithmic predictions on human expert judgment. Leveraging an online survey of medical providers and data from a leading U.S. hospital, we develop a ML algorithm and compare its performance with that of medical experts in the task of predicting 30-day readmissions after solid-organ transplantation. We find that our algorithm is not only more accurate in predicting clinical risk but can also positively influence human judgment. However, its potential impact is mediated by the users’ degree of algorithm aversion and trust. We show that, while our ML algorithm establishes non-linear associations between patient characteristics and the outcome of interest, human experts mostly attribute risk in a linear fashion. To capture potential synergies between human experts and the algorithm, we propose a human-algorithm “centaur” model. We show that it is able to outperform human experts and the best ML algorithm by systematically enhancing algorithmic performance with human-based intuition. Our results suggest that implementing the centaur model could reduce the average patient readmission rate by 26.4%, yielding up to a 770k dollar reduction in annual expenditure at our partner hospital and up to $67 million savings in overall U.S. healthcare expenditures.

A main research goal in various studies is to use an observational data set and provide a new set of counterfactual guidelines that can yield causal improvements. Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process and enable researchers to find guidelines that are both personalized and dynamic. However, available methods in finding optimal DTRs often rely on assumptions that are violated in real-world applications (e.g., medical decision-making or public policy), especially when (a) the existence of unobserved confounders cannot be ignored, and (b) the unobserved confounders are time-varying (e.g., affected by previous actions). When such assumptions are violated, one often faces ambiguity regarding the underlying causal model that is needed to be assumed to obtain an optimal DTR. This ambiguity is inevitable, since the dynamics of unobserved confounders and their causal impact on the observed part of the data cannot be understood from the observed data. Motivated by a case study of finding superior treatment regimes for patients who underwent transplantation in our partner hospital and faced a medical condition known as New Onset Diabetes After Transplantation (NODAT), we extend DTRs to a new class termed Ambiguous Dynamic Treatment Regimes (ADTRs), in which the casual impact of treatment regimes is evaluated based on a “cloud” of potential causal models. We then connect ADTRs to Ambiguous Partially Observable Mark Decision Processes (APOMDPs) proposed by Saghafian (2018), and consider unobserved confounders as latent variables but with ambiguous dynamics and causal effects on observed variables. Using this connection, we develop two Reinforcement Learning methods termed Direct Augmented V-Learning (DAV-Learning) and Safe Augmented V-Learning (SAV-Learning), which enable using the observed data to efficiently learn an optimal treatment regime. We establish theoretical results for these learning methods, including (weak) consistency and asymptotic normality. We further evaluate the performance of these learning methods both in our case study (using clinical data) and in simulation experiments (using synthetic data). We find promising results for our proposed approaches, showing that they perform well even compared to an imaginary oracle who knows both the true causal model (of the data generating process) and the optimal regime under that model.

Various hospitals in the U.S. and around the world suffer from the well-known problem of Emergency Department (ED) overcrowding, which prevents them from serving their patients in effective and efficient ways. An important contributor to this problem, which became even more dire after the COVID-19 pandemic, is prolonged boarding of patients who are admitted to inpatient units through the ED. Patients admitted through the ED constitute about 50% of all non-obstetrical hospital admissions in the U.S., and may be boarded in the ED for long hours with the hope of finding an available bed in their primary inpatient unit. In this chapter, we shed light on effective ways of reducing ED boarding times by considering the trade-off between keeping patients in the ED and assigning them to a secondary inpatient unit. The former can increase the risk of adverse events and also cause congestion in the ED (which, in turn, prevents from serving new ED patients in a timely manner), whereas the latter may adversely impact the quality of care post ED service. Further complicating this calculus is the fact that a secondary inpatient unit for a currently boarded ED patient can be the primary unit for a future arriving patient; assignments, therefore, should be made in an orchestrated way. Developing a queueing-based Markov decision process, we demonstrate that patience in transferring patients is a virtue, but only up to a point. We also find that, contrary to the prevalent perception, idling inpatient beds in hospitals can be beneficial (under some circumstances). Since the optimal policy for dynamically assigning patients to their primary and secondary inpatient units is complex and hard to implement in hospitals, we develop a simple policy which we term penalty-adjusted Largest Expected Workload Cost (LEWC-p). Using simulation analyses calibrated with hospital data, we find that implementing this policy could significantly help hospitals to improve their patient safety by reducing boarding times while controlling the overflow of patients to secondary units. Using data analyses and various simulation experiments, we also help hospital administrators by generating insights into hospital conditions under which achievable improvements are significant.

In this article, we focus on recent advancements in Generative AI, and especially in Large Language Models (LLMs). We first present a framework that allows understanding the core characteristics of centaurs. We argue that symbiotic learning and incorporation of human intuition are two main characteristics of centaurs that distinguish them from other models in Machine Learning (ML) and AI. Using these core characteristics, we also present a few specific methods of creating centaurs. We then argue that the growth and success of LLMs are to a great extent due to the fact that they are moved from pure ML algorithms to human-algorithm centaurs. We present various evidence to demonstrate this, particularly by focusing on the advantages of the so-called “fine-tuning” approaches such as the Reinforcement Learning with Human Feedback (RLHF) method used in various LLMs (e.g., OpenAI’s GPT-4, Antropic’s Claude, Google’s Bard, and Meta’s LLaMA 2-Chat). We also discuss evidence showing that these fine-tuning approaches can turn Generative AI tools into cognitive models, capable of representing human behavior. In addition, we elaborate on three main advantages of centaurs: removing barriers with respect to algorithm aversion, huma aversion, and casual aversion. We then briefly conclude by discussing two main points: (1) recent advancements in creating centaurs have moved us closer to reaching the goals that the founding fathers of AI—John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon—stated in 1955 as part of their proposed 2-month, 10-man study of AI to be held at Dartmouth; and (2) the future of AI development and use in many domains will most likely need to focus on centaurs as opposed to other traditional approaches in ML and AI.

The U.S. healthcare system is undergoing a period of substantial change, with hospitals purchasing many physician practices (\vertical integration"). In theory, this vertical integration could improve quality by promoting care coordination, but could also worsen it by impacting the care delivery patterns. The evidence quantifying these effects is limited, because of the lack of understanding of how physicians' behaviors alter in response to the changes in financial ownership and incentive structures of the integrated organizations. We study the impact of vertical integration by examining Medicare patients treated by gastroenterologists, a specialty with a large outpatient volume, and a recent increase in vertical integration. Using a causal model and large-scale patient-level national panel data that includes 2.6 million patient visits across 5,488 physicians, we examine changes in various measures of care delivery. We nd that physicians signi ficantly alter their care process (e.g., in using anesthesia with deep sedation) after they vertically integrate, and there is a substantial increase in patients' post-procedure complications. We further provide evidence that the financial incentive structure of the integrated practices is the main reason for the changes in physician behavior, since it discourages the integrated practices from allocating expensive resources to relatively unprofi table procedures. We also nd that although integration improves operational efficiency (e.g., measured by physicians' throughput), it negatively affects quality and overall spending. Finally, to shed light on potential mechanisms through which policymakers can mitigate the negative consequences of vertical integration, we perform both mediation and cost-effectiveness analyses, and highlight some useful policy levers.

Emergency Department (ED) patients may receive varying diagnostic workups and dispositions based on physician factors instead of solely based on presenting conditions. This study delves into the contrasting practices of batch-ordering multiple tests simultaneously versus the sequential ordering of tests based on previous results. Our analysis revealed stark differences in physician diagnostic approaches, even when working in similar environments. Findings suggest that physicians who predominantly make use of batching (“batchers”) tend to order more tests, which is associated with longer lengths of stay and increased costs. In contrast, other physicians (“non-batchers”) order fewer tests, which is associated with lower lengths of stay and costs, without any impact on primary ED outcome measures, such as the 72-hour rate of return. Thus, our results suggest an “information gain” advantage in the non-batching strategy: by ordering sequentially, non-batchers obtain the diagnostic information needed with a lower number of tests, enabling them to deliver the same quality of care more efficiently (e.g., with a lower length of stay and cost) than batchers. Finally, our study shows that the decision to batch order diagnostic tests can be optimized for each patient using a few variables, including acuity, chief complaints, and the ED volume at arrival.

In various organizations including hospitals, individuals are not forced to follow specific assignments, and thus, deviations from preferred task assignments are common. This is due to the conventional wisdom that professionals should be given the flexibility to deviate from preferred assignments as needed. It is unclear, however, whether and when this conventional wisdom is true. We use evidence on the assignments of generalist and specialists to patients in our partner hospital (a children’s hospital), and generate insights into whether and when hospital administrators should disallow such flexibility. We do so by identifying 73 top medical diagnoses and using detailed patient-level electronic medical record (EMR) data of more than 4,700 hospitalizations. In parallel, we conduct a survey of medical experts and utilize it to identify the preferred provider type that should have been assigned to each patient. Using these two sources of data, we examine the consequence of deviations from preferred provider assignments on three sets of performance measures: operational efficiency (measured by length of stay), quality of care (measured by 30-day readmissions and adverse events), and cost (measured by total charges). We find that deviating from preferred assignments is beneficial for task types (patients’ diagnosis in our setting) that are either (a) well-defined (improving operational efficiency and costs), or (b) require high contact (improving costs and adverse events, though at the expense of lower operational efficiency). For other task types (e.g., highly complex or resource-intensive tasks), we observe that deviations are either detrimental or yield no tangible benefits, and thus, hospitals should try to eliminate them (e.g., by developing and enforcing assignment guidelines). To understand the causal mechanism behind our results, we make use of mediation analysis and find that utilizing advanced imaging (e.g., MRIs, CT scans, or nuclear radiology) plays an important role in how deviations impact performance outcomes. Our findings also provide evidence for a “no free lunch” theorem: while for some task types deviations are beneficial regarding some performance measures, they can simultaneously degrade performance in terms of other dimensions. To provide clear recommendations for hospital administrators, we also consider counterfactual scenarios corresponding to imposing the preferred assignments fully or partially, and perform cost-effectiveness analyses. Our results indicate that enforcing the preferred assignments either for all tasks or only for resource-intensive tasks is cost-effective, with the latter being the superior policy. Finally, by comparing deviations during weekdays and weekends, early shifts and late shifts, and high congestion and low congestion periods, our results shed light on some environmental conditions under which deviations occur more in practice.

As a major public health crisis, the opioid epidemic caused over 556,000 deaths in the U.S. between 2000 and 2020. To control the epidemic, the Centers for Disease Control and Prevention (CDC) has developed some general guidelines, encouraging physicians to use opioid medications only when their benefits outweigh their risks. The CDC’s 2016 guidelines mainly left it to physicians to decide when the benefits outweigh the risks. A few years later (in 2022), the CDC made some modifications to make its recommendations a bit less reliant on each individual physician’s perception of benefits versus risks. In complex and high stake decision-making environments such as those pertaining the use of opioid medications, it is not clear whether and how human-based perceptions might differ from algorithmic-based ones. In this study, we first develop some longitudinal machine learning algorithms (e.g., historical random forest, recurrent neural networks, and long short-term memory networks) and train them on clinical evidence of more than 3 million patients. We then feed the best machine learning algorithm to a mathematical model that enables determining cost-effective treatments for each patient in a personalized manner. Through extensive numerical experiments, we compare the treatment options and recommendations from our algorithmic-based approach with human-based ones that are currently followed in the medical practice. Compared to the human-based approach, our results show that the average saving in quality-adjusted life years and costs obtained by following our algorithmic-based treatments are about 2.82 days and $461.46 per patient per year. Finally, we make use of our findings and generate insights for policymakers as well as individual physicians into better ways of managing opioid prescriptions (and hence, the opioid epidemic) by incorporating and interacting with our algorithmic-based approach.

Objective: To describe the relationship between emergency department resource utilization and admission rate at the level of the individual physician. Methods: Retrospective observational study of physician resource utilization and admitting data at two emergency departments. We calculated observed to expected (O/E) ratios for four measures of resource utilization (intravenous medications and fluids, laboratory testing, plain radiographs, and advanced imaging studies) as well as for admission rate. Expected values reflect adjustment for patient- and time-based variables. We compared O/E ratios for each type of resource utilization to the O/E ratio for admission for each provider. We report degree of correlation (slope of the trendline) and strength of correlation (adjusted R2 value) for each association, as well as categorical results after clustering physicians based on the relationship of resource utilization to admission rate. Results: There were statistically significant positive correlations between resource utilization and physician admission rate. Physicians with lower resource utilization rates were more likely to have lower admission rates, and those with higher resource utilization rates were more likely to have higher admission rates. Conclusions: In a two-facility study, emergency physician resource utilization and admission rate were positively correlated: those who used more ED resources also tended to admit more patients. These results add to a growing understanding of emergency physician variability.