Sparser Johnson-Lindenstrauss Transforms

Abstract

We give two different Johnson-Lindenstrauss distributions, each with column sparsity (s = \Theta(\epsilon^{−1} log(1/\delta))) and embedding into optimal dimension (k = O(\epsilon^{−2} log(1/\delta))) to achieve distortion (1\pm \epsilon) with probability (1−\delta). That is, only an (O(\epsilon))-fraction of entries are non-zero in each embedding matrix in the supports of our distributions. These are the first distributions to provide o(k) sparsity for all values of (\epsilon), (\delta). Previously the best known construction obtained (s = \overset \sim \Theta (\epsilon^{-1} log^2(1/\delta))^1) [Dasgupta-Kumar-Sarlós, STOC 2010]. In addition, one of our distributions can be sampled from a seed of (O(log(1/\delta) log d)) uniform random bits. Some applications that use Johnson-Lindenstrauss embeddings as a black box, such as those in approximate numerical linear algebra ([Sarlós, FOCS 2006], [Clarkson-Woodruff, STOC 2009]), require exponentially small (\delta). Our linear dependence on (log(1/\delta)) in the sparsity is thus crucial in these applications to obtain speedup.