TY - JOUR
T1 - Searching for Unknown Anomalies in Hierarchical Data Streams
AU - Gafni, Tomer
AU - Cohen, Kobi
AU - Zhao, Qing
N1 - Funding Information: Manuscript received June 20, 2021; revised August 15, 2021; accepted August 16, 2021. Date of publication August 20, 2021; date of current version September 14, 2021. The work of Tomer Gafni and Kobi Cohen was supported in part by the Cyber Security Research Center at Ben-Gurion University of the Negev, and in part by U.S.-Israel Binational Science Foundation (BSF) under Grant 2017723. The work of Qing Zhao was supported in part by the National Science Foundation under Grant CCF1815559. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. John Ball. (Corresponding author: Tomer Gafni.) Tomer Gafni and Kobi Cohen are with the School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel (e-mail: gafnito@post.bgu.ac.il; kobi.cohen10@gmail.com). Publisher Copyright: © 1994-2012 IEEE.
PY - 2021/1/1
Y1 - 2021/1/1
N2 - We consider the problem of anomaly detection among a large number of processes, where the probabilistic models of anomalies are unknown. At each time, aggregated noisy observations can be taken from a chosen subset of processes, where the chosen subset conforms to a tree structure. The observation distribution depends on the chosen subset and the absence/presence of anomalies. We develop a sequential search strategy using a hierarchical Kolmogorov-Smirnov (KS) statistics. Referred to as Tree-based Anomaly Search using KS statistics (TASKS), the proposed strategy is order-optimal with respect to the size of the search space and the detection accuracy.
AB - We consider the problem of anomaly detection among a large number of processes, where the probabilistic models of anomalies are unknown. At each time, aggregated noisy observations can be taken from a chosen subset of processes, where the chosen subset conforms to a tree structure. The observation distribution depends on the chosen subset and the absence/presence of anomalies. We develop a sequential search strategy using a hierarchical Kolmogorov-Smirnov (KS) statistics. Referred to as Tree-based Anomaly Search using KS statistics (TASKS), the proposed strategy is order-optimal with respect to the size of the search space and the detection accuracy.
KW - Anomaly detection
KW - dynamic search
KW - sequential design of experiments
UR - http://www.scopus.com/inward/record.url?scp=85113329166&partnerID=8YFLogxK
U2 - https://doi.org/10.1109/LSP.2021.3106587
DO - https://doi.org/10.1109/LSP.2021.3106587
M3 - Article
SN - 1070-9908
VL - 28
SP - 1774
EP - 1778
JO - IEEE Signal Processing Letters
JF - IEEE Signal Processing Letters
ER -