DeepStream: Autoencoder-based stream temporal clustering and anomaly detection

Shimon Harush, Yair Meidan, Asaf Shabtai

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

The increasing number of IoT devices in “smart” environments, such as homes, offices, and cities, produce seemingly endless data streams and drive many daily decisions. Consequently, there is growing interest in identifying contextual information from sensor data to facilitate the performance of various tasks, e.g., traffic management, cyber attack detection, and healthcare monitoring. The correct identification of contexts in data streams is helpful for many tasks, for example, it can assist in providing high-quality recommendations to end users and in reporting anomalous behavior based on the detection of unusual contexts. This paper presents DeepStream, a novel data stream temporal clustering algorithm that dynamically detects sequential and overlapping clusters. DeepStream is tuned to classify contextual information in real time and is capable of coping with a high-dimensional feature space. DeepStream utilizes stacked autoencoders to reduce the dimensionality of unbounded data streams and for cluster representation. This method detects contextual behavior and captures nonlinear relations of the input data, giving it an advantage over existing methods that rely on PCA. We evaluated DeepStream empirically using four sensor and IoT datasets and compared it to five state-of-the-art stream clustering algorithms. Our evaluation shows that DeepStream outperforms all of these algorithms. Our evaluation also demonstrates how DeepStream's improved clustering performance results in improved detection of anomalous data.

Original languageEnglish
Article number102276
JournalComputers and Security
Volume106
DOIs
StatePublished - 1 Jul 2021

Keywords

  • Activity recognition
  • Anomaly detection
  • Autoencoder
  • Dimensionality reduction
  • Stream clustering

ASJC Scopus subject areas

  • General Computer Science
  • Law

Fingerprint

Dive into the research topics of 'DeepStream: Autoencoder-based stream temporal clustering and anomaly detection'. Together they form a unique fingerprint.

Cite this