Enhanced situation space mining for data streams

Yisroel Mirsky, Tal Halpern, Rishabh Upadhyay, Sivan Toledo, Yuval Elovici

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Data streams can capture the situation which an actor is experiencing. Knowledge of the present situation is highly beneficial for a wide range of applications. An algorithm called pcStream can be used to extract situations from a numerical data stream in an unsupervised manner. Although pcStream outperforms other stream clustering algorithms at this task, pcStream has two major flaws. The first is its complexity due to continuously performing principal component analysis (PCA). The second is its difficulty in detecting emerging situations whose distributions overlap in the same feature space. In this paper we introduce pcStream2, a variant of pcStream which employs windowing and persistence in order to distinguish between emerging overlapping concepts. We also propose the use of incremental PCA (IPCA) to reduce the overall complexity and memory requirements of the algorithm. Although any IPCA algorithm can be used, we use a novel IPCA algorithm called Just-In-Time PCA which is better suited for processing streams. JIT-PCA makes intelligent 'short cuts' in order to reduce computations. We provide experimental results on real-world datasets that demonstrates how the proposed improvements make pcStream2 a more accurate and practical tool for situation space mining.

Original languageAmerican English
Title of host publication32nd Annual ACM Symposium on Applied Computing, SAC 2017
Pages842-849
Number of pages8
ISBN (Electronic)9781450344869
DOIs
StatePublished - 3 Apr 2017
Event32nd Annual ACM Symposium on Applied Computing, SAC 2017 - Marrakesh, Morocco
Duration: 4 Apr 20176 Apr 2017

Publication series

NameProceedings of the ACM Symposium on Applied Computing
VolumePart F128005

Conference

Conference32nd Annual ACM Symposium on Applied Computing, SAC 2017
Country/TerritoryMorocco
CityMarrakesh
Period4/04/176/04/17

Keywords

  • Context space theory
  • Data mining
  • Data stream

All Science Journal Classification (ASJC) codes

  • Software

Fingerprint

Dive into the research topics of 'Enhanced situation space mining for data streams'. Together they form a unique fingerprint.

Cite this