Benchmarking File System Benchmarking: It *IS* Rocket Science

DSpace/Manakin Repository

Benchmarking File System Benchmarking: It *IS* Rocket Science

Citable link to this page

. . . . . .

Title: Benchmarking File System Benchmarking: It *IS* Rocket Science
Author: Seltzer, Margo I.; Tarasov, Vasily; Bhanage, Saumitra; Zadok, Erez

Note: Order does not necessarily reflect citation order of authors.

Citation: Tarasov, Vasily, Saumitra Bhanage, Erez Zadok and Margo Seltzer. Forthcoming. Benchmarking file system benchmarking: It *IS* rocket science. In Proceedings of the 13th Workshop on Hot Topics in Operating Systems (HotOS XIII). May 9-11, 2011, Napa, California. Berkeley, CA: USENIX Association.
Full Text & Related Files:
Abstract: The quality of file system benchmarking has not improved in over a decade of intense research spanning hundreds of publications. Researchers repeatedly use a wide range of poorly designed benchmarks, and in most cases, develop their own ad-hoc benchmarks. Our community lacks a definition of what we want to benchmark in a file system. We propose several dimensions of file system benchmarking and review the wide range of tools and techniques in widespread use. We experimentally show that even the simplest of benchmarks can be fragile, producing performance results spanning orders of magnitude. It is our hope that this paper will spur serious debate in our community, leading to action that can improve how we evaluate our file and storage systems.
Published Version: http://www.usenix.org/events/hotos11/tech/final_files/Tarasov.pdf
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAP
Citable link to this page: http://nrs.harvard.edu/urn-3:HUL.InstRepos:5168854

Show full Dublin Core record

This item appears in the following Collection(s)

  • FAS Scholarly Articles [6463]
    Peer reviewed scholarly articles from the Faculty of Arts and Sciences of Harvard University
 
 

Search DASH


Advanced Search
 
 

Submitters