Core Concepts
The core message of this paper is to provide optimal communication protocols for approximating sums of entrywise functions in the coordinator model, as well as efficient algorithms for solving linear algebra problems in more general network topologies using the personalized CONGEST model.
Abstract
The paper addresses the problem of approximating the sum Σi f(xi) in the coordinator model, where each server j holds a non-negative vector x(j) and the goal is to approximate the sum Σi f(xi) up to a 1±ε factor, for a given non-negative function f.
The key highlights and insights are:
The authors introduce a new parameter cf[s] that captures the communication complexity of approximating Σi f(xi) more accurately than the previously used parameter cf,s.
For functions f that satisfy an "approximate invertibility" property, the authors provide a two-round protocol that uses Oθ,θ',θ''(cf[s]/ε^2) bits of communication to approximate Σi f(xi) up to a 1±ε factor.
For the special case of f(x) = x^k, the authors show that cf[s] = s^(k-1) and their protocol matches the known lower bounds, resolving the open question of the optimal communication complexity of Fk-moment estimation in the coordinator model.
The authors also show that any one-round algorithm for Fk-moment estimation must use Ω(s^(k-1)/ε^k) bits of communication, demonstrating the optimality of their two-round protocol.
Beyond the coordinator model, the authors study the personalized CONGEST model and provide efficient algorithms for computing ℓp-subspace embeddings, solving ℓp-regression, and low-rank approximation, where the communication per node in each round is polylogarithmic in the relevant parameters.
Stats
Σi f(xi) = Σi (Σj xi(j))
cf[s] = smallest number such that f(Σj yj) ≤ cf[s] * (Σj √f(yj))^2 for all yj ≥ 0
Ω(s^(k-1)/ε^2) lower bound for Fk-moment estimation
Quotes
"For this broad class of functions, our result improves upon the communication bounds achieved by Kannan, Vempala, and Woodruff (COLT 2014) and Woodruff and Zhang (STOC 2012), obtaining the optimal communication up to polylogarithmic factors in the minimum number of rounds."
"We show that our protocol can also be used for approximating higher-order correlations."
"Our sketch construction may be of independent interest and can implement any importance sampling procedure that has a monotonicity property."