Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes
Open/View files
Date
February 23, 2018
Author(s)
Krusche, Peter
Trigg, Len
Boutros, Paul C
Mason, Christopher E
De La Vega, Francisco M
Moore, Benjamin L
Gonzalez-Porta, Mar
Eberle, Michael A
Tezak, Zivana
Labadibi, Samir
Truty, Rebecca
Asimenos, George
Funke, Birgit
Fleharty, Mark
Salit, Marc
Zook, Justin M
Publisher
Cold Spring Harbor Laboratory
The Harvard community has made this article openly available. Please share how this access benefits you.
Citation
Krusche, Peter, Len Trigg, Paul C Boutros, Christopher E Mason, Francisco M De La Vega, Benjamin L Moore, Mar Gonzalez-Porta, et al. “Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes.” BioRxiv, May 24, 2018. https://doi.org/10.1101/270157.
Abstract
Standardized benchmarking methods and tools are essential to robust accuracy assessment of NGS variant calling. Benchmarking variant calls requires careful attention to definitions of performance metrics, sophisticated comparison approaches, and stratification by variant type and genome context. To address these needs, the Global Alliance for Genomics and Health (GA4GH) Benchmarking Team convened representatives from sequencing technology developers, government agencies, academic bioinformatics researchers, clinical laboratories, and commercial technology and bioinformatics developers for whom benchmarking variant calls is essential to their work. This team addressed challenges in (1) matching variant calls with different representations, (2) defining standard performance metrics, (3) enabling stratification of performance by variant type and genome context, and (4) developing and describing limitations of high-confidence calls and regions that can be used as "truth". Our methods are publicly available on GitHub (https://github.com/ga4gh/benchmarking-tools) and in a web-based app on precisionFDA, which allow users to compare their variant calls against truth sets and to obtain a standardized report on their variant calling performance. Our methods have been piloted in the precisionFDA variant calling challenges to identify the best-in-class variant calling methods within high-confidence regions. Finally, we recommend a set of best practices for using our tools and critically evaluating the results.
Terms of Use
Metadata Only
Project(s)
Nature Biotechnology
Collection