HyperFork: Improving Serverless Latency and Throughput Through Virtual Machine Flash-Cloning

Abstract

Serverless computing provides a simple model for executing computations in response to triggers. In many serverless deployments, these computations are executed within virtual machines for stronger isolation and fairness guarantees. As a result, the time required to boot a new virtual machine is a significant contributor to the latency of a serverless application. To minimize the latency introduced by the boot process, we propose an alternative paradigm in which new virtual machines are launched by flash-cloning a pre-booted machine image. To assess this scheme, we have created HyperFork, a flash-clone implementation built on top of KVM and the kvmtool virtual machine monitor. HyperFork duplicates the full machine state, including memory, disk contents, and CPU state, to the new virtual machine. For the contents of memory and disk, our implementation leverages copy-on-write optimizations exposed by the Linux kernel. We demonstrate that HyperFork reduces virtual machine cold-start times by up to 98.4% while increasing throughput by 46%-502% in serverless workloads. We discuss the optimizations employed to achieve this performance along with security and correctness implications of our approach.