Core Concepts
There exists a deterministic streaming algorithm for ε-approximate quantile sketching that uses O(ε^-1) words of space, resolving a long-standing open problem.
Abstract
The content presents a deterministic streaming algorithm for ε-approximate quantile sketching that uses O(ε^-1) words of space, improving upon the previously best-known algorithms.
Key highlights:
The algorithm goes beyond the comparison-based lower bound, which was the previous state-of-the-art. This is achieved by exploiting the fact that the elements come from a bounded universe.
The algorithm uses a recursive structure based on the q-digest data structure, with several optimizations to reduce the space complexity to the optimal O(ε^-1) words.
The algorithm handles insertions by batching them and compressing the data into the lower layers of the recursive structure. This allows maintaining the invariant of having only full or empty nodes, except at the last layer.
The error analysis shows that the rank estimates have an additive error of at most εt, where t is the current stream size.
The algorithm also achieves optimal time complexity of O(log(1/ε)) amortized time per operation, under mild assumptions.
The content provides a detailed technical overview, describes the data structure and algorithms, analyzes the error and complexity, and discusses further directions and optimality of the solution.