TY - JOUR
T1 - Randomized admission policy for efficient top-k, frequency, and volume estimation
AU - Ben Basat, Ran
AU - Chen, Xiaoqi
AU - Einziger, Gil
AU - Friedman, Roy
AU - Kassner, Yaron
N1 - Funding Information:
Manuscript received November 9, 2018; revised April 16, 2019; accepted May 20, 2019; approved by IEEE/ACM TRANSACTIONS ON NETWORKING Editor A. Dainotti. Date of publication June 10, 2019; date of current version August 16, 2019. This work was supported in part by MOST under Grant #3-10886, in part by ISF under Grant #1505/16, in part by the Technion-HPI Research School, in part by the Zuckerman Institute, in part by the Technion Hiroshi Fujiwara Cyber Security Research Center, in part by the Israel Cyber Directorate, and in part by the Cyber Security Research Center at Ben-Gurion University. (Corresponding author: Gil Einziger.) R. Ben Basat is with the Department of Computer Science, Harvard University, Cambridge, MA 02138 USA.
Publisher Copyright:
© 2019 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.
PY - 2019/8/1
Y1 - 2019/8/1
N2 - Network management protocols often require timely and meaningful insight about per flow network traffic. This paper introduces Randomized Admission Policy RAP -a novel algorithm for the frequency, top-k, and byte volume estimation problems, which are fundamental in network monitoring. We demonstrate space reductions compared to the alternatives, for the frequency estimation problem, by a factor of up to 32 on real packet traces and up to 128 on heavy-tailed workloads. For top-$k$ identification, RAP exhibits memory savings by a factor of between 4 and 64 depending on the workloads' skewness. These empirical results are backed by formal analysis, indicating the asymptotic space improvement of our probabilistic admission approach. In Addition, we present d-way RAP, a hardware friendly variant of RAP that empirically maintains its space and accuracy benefits.
AB - Network management protocols often require timely and meaningful insight about per flow network traffic. This paper introduces Randomized Admission Policy RAP -a novel algorithm for the frequency, top-k, and byte volume estimation problems, which are fundamental in network monitoring. We demonstrate space reductions compared to the alternatives, for the frequency estimation problem, by a factor of up to 32 on real packet traces and up to 128 on heavy-tailed workloads. For top-$k$ identification, RAP exhibits memory savings by a factor of between 4 and 64 depending on the workloads' skewness. These empirical results are backed by formal analysis, indicating the asymptotic space improvement of our probabilistic admission approach. In Addition, we present d-way RAP, a hardware friendly variant of RAP that empirically maintains its space and accuracy benefits.
KW - Algorithm design and analysis
KW - Approximation algorithms
UR - http://www.scopus.com/inward/record.url?scp=85074833448&partnerID=8YFLogxK
U2 - 10.1109/TNET.2019.2918929
DO - 10.1109/TNET.2019.2918929
M3 - Article
AN - SCOPUS:85074833448
SN - 1063-6692
VL - 27
SP - 1432
EP - 1445
JO - IEEE/ACM Transactions on Networking
JF - IEEE/ACM Transactions on Networking
IS - 4
M1 - 3370594
ER -