File Layout and File System Performance

Abstract

Most contemporary implementations of the Berkeley Fast File System optimize file system throughput by allocating logically sequential data to physically contiguous disk blocks. This clustering is effective when there are many contiguous free blocks on the file system. But the repeated creation and deletion of files of varying sizes that occurs over time on active file systems is likely to cause fragmentation of free space, limiting the ability of the file system to allocate data contiguously and therefore degrading performance. This paper presents empirical data and the analysis of allocation and fragmentation in the SunOS 4.1.3 file system (a derivative of the Berkeley Fast File System). We have collected data from forty-eight file systems on four file servers over a period of ten months. Our data show that small files are more fragmented than large files, with fewer than 35% of the blocks in two block files being allocated optimally, but more than 80% of the blocks in files larger than 256 kilobytes being allocated optimally. Two factors are responsible for this difference in fragmentation, an uneven distribution of free space within file system cylinder groups and a disk allocation algorithm which frequently allocates the last block of a file discontiguously from the rest of the file. Performance measurements on replicas of active file systems show that they seldom perform as well as comparable empty file systems but that this performance degradation is rarely more than 10–15%. This decline in performance is directly correlated to the amount of fragmentation in the files used by the benchmark programs. Both file system utilization and the amount of fragmentation in existing files on the file system influence the amount of fragmentation in newly created files. Characteristics of the file system workload also have a significant impact of file system fragmentation and performance, with typical news server workloads causing extreme fragmentation.