Person:

Khorasani, Ramin

Background: Communication of critical results from diagnostic procedures between caregivers is a Joint Commission national patient safety goal. Evaluating critical result communication often requires manual analysis of voluminous data, especially when reviewing unstructured textual results of radiologic findings. Information retrieval (IR) tools can facilitate this process by enabling automated retrieval of radiology reports that cite critical imaging findings. However, IR tools that have been developed for one disease or imaging modality often need substantial reconfiguration before they can be utilized for another disease entity. Purpose: This paper: 1) describes the process of customizing two Natural Language Processing (NLP) and Information Retrieval/Extraction applications – an open-source toolkit, A Nearly New Information Extraction system (ANNIE); and an application developed in-house, Information for Searching Content with an Ontology-Utilizing Toolkit (iSCOUT) – to illustrate the varying levels of customization required for different disease entities and; 2) evaluates each application’s performance in identifying and retrieving radiology reports citing critical imaging findings for three distinct diseases, pulmonary nodule, pneumothorax, and pulmonary embolus. Results: Both applications can be utilized for retrieval. iSCOUT and ANNIE had precision values between 0.90-0.98 and recall values between 0.79 and 0.94. ANNIE had consistently higher precision but required more customization. Conclusion: Understanding the customizations involved in utilizing NLP applications for various diseases will enable users to select the most suitable tool for specific tasks.

Introduction: While only 15–20% of patients with foot and ankle injuries presenting to urgent care centers have clinically significant fractures, most undergo radiography. We examined the impact of electronic point-of-care clinical decision support (CDS) on adherence to the Ottawa Ankle Rules (OAR), as well as use and yield of foot and ankle radiographs in patients with acute ankle injury. Methods: We obtained institutional review board approval for this randomized controlled study performed April 18, 2012—December 15, 2013. All ordering providers credentialed at an urgent care affiliated with a quaternary care academic hospital were randomized to either receive or not receive CDS, based on the OAR and integrated into the physician order-entry system, with feedback at the time of imaging order. If the patient met OAR low-risk criteria, providers were advised against imaging and could either cancel the order or ignore the alert. We identified patients with foot and ankle complaints via ICD-9 billing codes and electronic health records and radiology reports reviewed for those who were eligible. Chi-square was used to compare adherence to the OAR (primary outcome), radiography utilization rate and radiography yield of foot and ankle imaging (secondary outcomes) between the intervention and control groups. Results: Of 14,642 patients seen at urgent care during the study period, 613 (4.2%, representing 632 visits) presented with acute ankle injury and were eligible for application of the OAR; 374 (59.2%) of these were seen by control-group providers. In the intervention group, CDS adherence was higher for both ankle (239/258=92.6% vs. 231/374=61.8%, p=0.02) and foot radiography (209/258=81.0% vs. 238/374=63.6%; p<0.01). However, ankle radiography use was higher in the intervention group (166/258=64.3% vs. 183/374=48.9%; p<0.01), while foot radiography use (141/258=54.6% vs. 202/374=54.0%; p=0.95) was not. Radiography yield was also higher in the intervention group (26/307=8.5% vs. 18/385=4.7%; p=0.04). Conclusion: Clinical decision support, previously demonstrated to improve guideline adherence for high-cost imaging, can also improve guideline adherence for radiography – as demonstrated by increased OAR adherence and increased imaging yield.

Objective For health information technology tools to fully inform evidence-based decisions, recommendations must be reliably assessed for quality and strength of evidence. We aimed to create an annotation framework for grading recommendations regarding appropriate use of diagnostic imaging examinations.

Methods The annotation framework was created by an expert panel (clinicians in three medical specialties, medical librarians, and biomedical scientists) who developed a process for achieving consensus in assessing recommendations, and evaluated by measuring agreement in grading the strength of evidence for 120 empirically selected recommendations using the Oxford Levels of Evidence.

Results Eighty-two percent of recommendations were assigned to Level 5 (expert opinion). Inter-annotator agreement was 0.70 on initial grading (κ = 0.35, 95% CI, 0.23-0.48). After systematic discussion utilizing the annotation framework, agreement increased significantly to 0.97 (κ = 0.88, 95% CI, 0.77-0.99).

Conclusions A novel annotation framework was effective for grading the strength of evidence supporting appropriate use criteria for diagnostic imaging exams.

Rationale and Objectives: Assess the impact of implementing a structured report template and a computer-aided diagnosis (CAD) tool on the quality of prostate multiparametric MRI (mp-MRI) reports. Materials and Methods: Institutional Review Board approval was obtained for this HIPAA-compliant study performed at an academic medical center. The study cohort included all prostate mp-MRI reports (n=385) finalized 6 months before and after implementation of a structured report template and a CAD tool (collectively the IT tools) integrated into the PACS workstation. Primary outcome measure was quality of prostate mp-MRI reports. An expert panel of our institution’s subspecialty trained abdominal radiologists defined prostate mp-MRI report quality as optimal, satisfactory or unsatisfactory based on documentation of 9 variables. Reports were reviewed to extract the predefined quality variables and determine whether the IT tools were used to create each report. Chi-square and Student’s t-tests were used to compare report quality before and after implementation of IT tools. Results: The overall proportion of optimal or satisfactory reports increased from 29.8% (47/158) to 53.3% (121/227) (p<0.001) after implementing the IT tools. While the proportion of optimal or satisfactory reports increased among reports generated using at least one of the IT tools (47/158=[29.8%] vs. 105/161=[65.2%]; p<0.001), there was no change in quality among reports generated without use of the IT tools (47/158=[29.8%] vs. 16/66=[24.2%]; p=0.404). Conclusion: The use of a structured template and CAD tool improved the quality of prostate mp-MRI reports compared to free-text report format and subjective measurement of contrast enhancement kinetic curve.

Background: Abdominal cross-sectional imaging is often performed to evaluate abnormal liver function test (LFT) results in hospitalized patients. However, few data are available regarding the yield and usefulness of imaging inpatients for the indication of abnormal LFT results, the process of requesting abdominal imaging studies, or the response to their findings. Methods: We retrospectively reviewed abdominal imaging scans that were obtained during a 27-month period. We matched the imaging studies done with the indication of abnormal LFT results; all scans were requested using computerized physician order entry. Reports were coded for interpretation and associated process step results. To determine the usefulness of the imaging studies, a random sample of patient charts with positively coded imaging studies were reviewed. Imaging examinations were considered useful if they provided new diagnostic information and/or changed subsequent patient care. Results: Of 6494 abdominal imaging studies, 856 were performed for the indication of abnormal LFT results and matched to both image reports and laboratory results. Report coding judged 37% of interpretations as clinically significant, including 27% with "positive" (abnormal results and explain the abnormal LFT results) examinations. Among the positive examinations, the most common diagnoses were biliary obstruction (25%), cholecystitis (21%), malignancy (20%), and cirrhosis (14%). Positively coded reports provided new clinical information in 63% of these studies and changed patient care in 42% of cases. Process measures assessed provision of additional information to and from radiologists (69% and 8%, respectively) and the frequency with which the findings of current abdominal imaging studies were compared with those of prior studies (59%). Conclusion: Abdominal cross-sectional imaging studies performed on inpatients with abnormal LFT results had a high diagnostic yield and frequently changed patient care.