Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2

View/ Open
Author
Mowery, Danielle L.
South, Brett R.
Christensen, Lee
Leng, Jianwei
Peltonen, Laura-Maria
Salanterä, Sanna
Suominen, Hanna
Martinez, David
Velupillai, Sumithra
Elhadad, Noémie
Pradhan, Sameer
Chapman, Wendy W.
Note: Order does not necessarily reflect citation order of authors.
Published Version
https://doi.org/10.1186/s13326-016-0084-yMetadata
Show full item recordCitation
Mowery, D. L., B. R. South, L. Christensen, J. Leng, L. Peltonen, S. Salanterä, H. Suominen, et al. 2016. “Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2.” Journal of Biomedical Semantics 7 (1): 43. doi:10.1186/s13326-016-0084-y. http://dx.doi.org/10.1186/s13326-016-0084-y.Abstract
Background: The ShARe/CLEF eHealth challenge lab aims to stimulate development of natural language processing and information retrieval technologies to aid patients in understanding their clinical reports. In clinical text, acronyms and abbreviations, also referenced as short forms, can be difficult for patients to understand. For one of three shared tasks in 2013 (Task 2), we generated a reference standard of clinical short forms normalized to the Unified Medical Language System. This reference standard can be used to improve patient understanding by linking to web sources with lay descriptions of annotated short forms or by substituting short forms with a more simplified, lay term. Methods: In this study, we evaluate 1) accuracy of participating systems’ normalizing short forms compared to a majority sense baseline approach, 2) performance of participants’ systems for short forms with variable majority sense distributions, and 3) report the accuracy of participating systems’ normalizing shared normalized concepts between the test set and the Consumer Health Vocabulary, a vocabulary of lay medical terms. Results: The best systems submitted by the five participating teams performed with accuracies ranging from 43 to 72 %. A majority sense baseline approach achieved the second best performance. The performance of participating systems for normalizing short forms with two or more senses with low ambiguity (majority sense greater than 80 %) ranged from 52 to 78 % accuracy, with two or more senses with moderate ambiguity (majority sense between 50 and 80 %) ranged from 23 to 57 % accuracy, and with two or more senses with high ambiguity (majority sense less than 50 %) ranged from 2 to 45 % accuracy. With respect to the ShARe test set, 69 % of short form annotations contained common concept unique identifiers with the Consumer Health Vocabulary. For these 2594 possible annotations, the performance of participating systems ranged from 50 to 75 % accuracy. Conclusion: Short form normalization continues to be a challenging problem. Short form normalization systems perform with moderate to reasonable accuracies. The Consumer Health Vocabulary could enrich its knowledge base with missed concept unique identifiers from the ShARe test set to further support patient understanding of unfamiliar medical terms.Other Sources
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4930590/pdf/Terms of Use
This article is made available under the terms and conditions applicable to Other Posted Material, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAACitable link to this page
http://nrs.harvard.edu/urn-3:HUL.InstRepos:27822158
Collections
- HMS Scholarly Articles [17301]
Contact administrator regarding this item (to report mistakes or request changes)