Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data

Or Dinari, Oren Freifeld

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Practical tools for clustering streaming data must be fast enough to handle the arrival rate of the observations. Typically, they also must adapt on the fly to possible lack of stationarity; i.e., the data statistics may be time-dependent due to various forms of drifts, changes in the number of clusters, etc. The
Dirichlet Process Mixture Model (DPMM), whose Bayesian nonparametric nature allows it to adapt its complexity to the data, seems a natural choice for the streaming-data case. In its classical formulation, however, the DPMM
cannot capture common types of drifts in the data statistics. Moreover, and regardless of that limitation, existing methods for online DPMM inference are too slow to handle rapid data streams. In this work we propose adapting both the DPMM and a known DPMM sampling-based non-streaming inference method for streaming-data clustering. We demonstrate the utility of the proposed
method on several challenging settings, where it obtains state-of-the-art results while being on par with other methods in terms of speed.
Original languageEnglish
Title of host publicationInternational Conference on Artificial Intelligence and Statistics (AISTATS)
Pages818-835
Number of pages18
Volume151
StatePublished - 2022

Fingerprint

Dive into the research topics of 'Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data'. Together they form a unique fingerprint.

Cite this