Distributed MCMC inference in dirichlet process mixture models using julia

Or Dinari, Angel Yu, Oren Freifeld, John Fisher

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

14 Scopus citations

Abstract

Due to the increasing availability of large data sets, the need for general-purpose massively-parallel analysis tools become ever greater. In unsupervised learning, Bayesian nonparametric mixture models, exemplified by the Dirichlet-Process Mixture Model (DPMM), provide a principled Bayesian approach to adapt model complexity to the data. Despite their potential, however, DPMMs have yet to become a popular tool. This is partly due to the lack of friendly software tools that can handle large datasets efficiently. Here we show how, using Julia, one can achieve efficient and easily-modifiable implementation of distributed inference in DPMMs. Particularly, we show how a recent parallel MCMC inference algorithm - originally implemented in C++ for a single multi-core machine - can be distributed efficiently across multiple multi-core machines using a distributed-memory model. This leads to speedups, alleviates memory and storage limitations, and lets us learn DPMMs from significantly larger datasets and of higher dimensionality. It also turned out that even on a single machine the proposed Julia implementation handles higher dimensions more gracefully (at least for Gaussians) than the original C++ implementation. Finally, we use the proposed implementation to learn a model of image patches and apply the learned model for image denoising. While we speculate that a highly-optimized distributed implementation in, say, C++ could have been faster than the proposed implementation in Julia, from our perspective as machine-learning researchers (as opposed to HPC researchers), the latter also offers a practical and monetary value due to the ease of development and abstraction level. Our code is publicly available at https://github.com/dinarior/dpmm subclusters.jl.

Original languageEnglish
Title of host publicationProceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019
PublisherInstitute of Electrical and Electronics Engineers
Pages518-525
Number of pages8
ISBN (Electronic)9781728109121
DOIs
StatePublished - 1 May 2019
Event19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019 - Larnaca, Cyprus
Duration: 14 May 201917 May 2019

Publication series

NameProceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019

Conference

Conference19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019
Country/TerritoryCyprus
CityLarnaca
Period14/05/1917/05/19

Keywords

  • Bayesian nonparametric mixture model
  • Data point
  • Dirichlet process
  • Image denoising
  • Image patch
  • Mixture model
  • Multiple multi core machine
  • Process mixture model
  • Single machine
  • Sub cluster
  • Sub cluster parameter
  • Sufficient statistic

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management
  • Management Science and Operations Research
  • Health Informatics

Fingerprint

Dive into the research topics of 'Distributed MCMC inference in dirichlet process mixture models using julia'. Together they form a unique fingerprint.

Cite this