Use of Outcomes to Evaluate Surveillance Systems for Bioterrorist Attacks

DSpace/Manakin Repository

Use of Outcomes to Evaluate Surveillance Systems for Bioterrorist Attacks

Citable link to this page


Title: Use of Outcomes to Evaluate Surveillance Systems for Bioterrorist Attacks
Author: McBrien, Kerry A; Abrams, Allyson M; Kleinman, Ken Paul; Prosser, Lisa Alison

Note: Order does not necessarily reflect citation order of authors.

Citation: McBrien, Kerry A., Ken P. Kleinman, Allyson M. Abrams, and Lisa A. Prosser. 2010. Use of outcomes to evaluate surveillance systems for bioterrorist attacks. BMC Medical Informatics and Decision Making 10: 25.
Full Text & Related Files:
Abstract: Background: Syndromic surveillance systems can potentially be used to detect a bioterrorist attack earlier than traditional surveillance, by virtue of their near real-time analysis of relevant data. Receiver operator characteristic (ROC) curve analysis using the area under the curve (AUC) as a comparison metric has been recommended as a practical evaluation tool for syndromic surveillance systems, yet traditional ROC curves do not account for timeliness of detection or subsequent time-dependent health outcomes. Methods: Using a decision-analytic approach, we predicted outcomes, measured in lives, quality adjusted life years (QALYs), and costs, for a series of simulated bioterrorist attacks. We then evaluated seven detection algorithms applied to syndromic surveillance data using outcomes-weighted ROC curves compared to simple ROC curves and timeliness-weighted ROC curves. We performed sensitivity analyses by varying the model inputs between best and worst case scenarios and by applying different methods of AUC calculation. Results: The decision analytic model results indicate that if a surveillance system was successful in detecting an attack, and measures were immediately taken to deliver treatment to the population, the lives, QALYs and dollars lost could be reduced considerably. The ROC curve analysis shows that the incorporation of outcomes into the evaluation metric has an important effect on the apparent performance of the surveillance systems. The relative order of performance is also heavily dependent on the choice of AUC calculation method. Conclusions: This study demonstrates the importance of accounting for mortality, morbidity and costs in the evaluation of syndromic surveillance systems. Incorporating these outcomes into the ROC curve analysis allows for more accurate identification of the optimal method for signaling a possible bioterrorist attack. In addition, the parameters used to construct an ROC curve should be given careful consideration.
Published Version: doi:10.1186/1472-6947-10-25
Other Sources:
Terms of Use: This article is made available under the terms and conditions applicable to Other Posted Material, as set forth at
Citable link to this page:
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)


Search DASH

Advanced Search