The paper introduces a new notion of multi-pass information complexity (MIC) and uses it to prove tight lower bounds for fundamental streaming problems that require multiple passes over the input.
For the coin problem, where the goal is to compute the majority of a stream of n i.i.d. uniform bits, the paper shows that any k-pass streaming algorithm requires Ω(log n/k) bits of memory to solve this problem with high probability. This significantly extends the previous Ω(log n) lower bound for single-pass algorithms.
For the needle problem, where the goal is to distinguish between a stream of n i.i.d. uniform samples and a stream where each item independently equals a randomly chosen "needle" with probability p, the paper shows a tight multi-pass lower bound of kps^2n = Ω(1), where s is the space used by the algorithm. This resolves an open question and improves upon the previous Ω(1/p^2n log n) lower bound.
The paper also presents applications of these multi-pass lower bounds to problems like approximate counting in strict turnstile streams, multi-ℓp-estimation, ℓ2-point query, ℓ2-heavy hitters, and sparse recovery in compressed sensing.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Mark Braverm... at arxiv.org 04-01-2024
https://arxiv.org/pdf/2403.20283.pdfDeeper Inquiries