Does Systems Research Measure Up?
Smith, KeithNote: Order does not necessarily reflect citation order of authors.
MetadataShow full item record
CitationSmall, Christopher, Narendra Ghosh, Hany Saleeb, Margo Selzter, and Keith Smith. 1997. Does Systems Research Measure Up? Harvard Computer Science Group Technical Report TR-16-97.
AbstractWe surveyed more than two hundred systems research papers published in the last six years, and found that, in experiment after experiment, systems researchers measure the same things, but in the majority of cases the reported results are not reproducible, comparable, or statistically rigorous. In this paper we present data describing the state of systems experimentation and suggest guidelines for structuring commonly run experiments, so that results from work by different researchers can be compared more easily. We conclude with recommendations on how to improve the rigor of published computer systems research.
Citable link to this pagehttp://nrs.harvard.edu/urn-3:HUL.InstRepos:25686817
- FAS Scholarly Articles