TY - GEN
T1 - Generalizing the layering method of indyk and Woodruff
T2 - 16th International Workshop on Approximation Algorithms for Combinatorial Optimization Problems, APPROX 2013 and the 17th International Workshop on Randomization and Computation, RANDOM 2013
AU - Braverman, Vladimir
AU - Ostrovsky, Rafail
PY - 2013/10/15
Y1 - 2013/10/15
N2 - In their ground-breaking paper, Indyk and Woodruff (STOC 05) showed how to compute the k-th frequency moment Fk (for k > 2) in space O(poly-log(n,m)·n1-2/k), giving the first optimal result up to poly-logarithmic factors in n and m (here m is the length of the stream and n is the size of the domain.) The method of Indyk and Woodruff reduces the problem of Fk to the problem of computing heavy hitters in the streaming manner. Their reduction only requires polylogarithmic overhead in term of the space complexity and is based on the fundamental idea of "layering". Since 2005 the method of Indyk and Woodruff has been used in numerous applications and has become a standard tool for streaming computations. We propose a new recursive sketch that generalizes and improves the reduction of Indyk and Woodruff. Our method works for any non-negative frequency-based function in several models, including the insertion-only model, the turnstile model and the sliding window model. For frequency-based functions with sublinear polynomial space complexity our reduction only requires log(c)(n) overhead, where log(c)(n) is the iterative log function. Thus, we improve the reduction of Indyk and Woodruff by polylogarithmic factor. We illustrate the generality of our method by several applications: frequency moments, frequency based functions, spatial data streams and measuring independence of data sets.
AB - In their ground-breaking paper, Indyk and Woodruff (STOC 05) showed how to compute the k-th frequency moment Fk (for k > 2) in space O(poly-log(n,m)·n1-2/k), giving the first optimal result up to poly-logarithmic factors in n and m (here m is the length of the stream and n is the size of the domain.) The method of Indyk and Woodruff reduces the problem of Fk to the problem of computing heavy hitters in the streaming manner. Their reduction only requires polylogarithmic overhead in term of the space complexity and is based on the fundamental idea of "layering". Since 2005 the method of Indyk and Woodruff has been used in numerous applications and has become a standard tool for streaming computations. We propose a new recursive sketch that generalizes and improves the reduction of Indyk and Woodruff. Our method works for any non-negative frequency-based function in several models, including the insertion-only model, the turnstile model and the sliding window model. For frequency-based functions with sublinear polynomial space complexity our reduction only requires log(c)(n) overhead, where log(c)(n) is the iterative log function. Thus, we improve the reduction of Indyk and Woodruff by polylogarithmic factor. We illustrate the generality of our method by several applications: frequency moments, frequency based functions, spatial data streams and measuring independence of data sets.
KW - Data streams
KW - frequencies
KW - recursion
KW - sketches
UR - https://www.scopus.com/pages/publications/84885214696
U2 - 10.1007/978-3-642-40328-6_5
DO - 10.1007/978-3-642-40328-6_5
M3 - Conference contribution
AN - SCOPUS:84885214696
SN - 9783642403279
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 58
EP - 70
BT - Approximation, Randomization, and Combinatorial Optimization
Y2 - 21 August 2013 through 23 August 2013
ER -