Collaborative inference via ensembles on the edge

Nir Shlezinger, Erez Farhan, Hai Morgenstern, Yonina C. Eldar

Research output: Contribution to journalConference articlepeer-review

16 Scopus citations


The success of deep neural networks (DNNs) as an enabler of artificial intelligence (AI) is heavily dependent on high computational resources. The increasing demands for accessible and personalized AI give rise to the need to operate DNNs on edge devices such as smartphones, sensors, and autonomous cars, whose computational powers are limited. Here we propose a framework for facilitating the application of DNNs on the edge in a manner which allows multiple users to collaborate during inference in order to improve their prediction accuracy. Our mechanism, referred to as edge ensembles, is based on having diverse predictors at each device, which can form a deep ensemble during inference. We analyze the latency induced in this collaborative inference approach, showing that the ability to improve performance via collaboration comes at the cost of a minor additional delay. Our experimental results demonstrate that collaborative inference via edge ensembles equipped with compact DNNs substantially improves the accuracy over having each user infer locally, and can outperform using a single centralized DNN larger than all the networks in the ensemble together.

Original languageEnglish
Pages (from-to)8478-8482
Number of pages5
JournalProceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing
StatePublished - 1 Jan 2021
Event2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada
Duration: 6 Jun 202111 Jun 2021


  • Deep ensembles
  • Edge computing
  • Neural networks

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering


Dive into the research topics of 'Collaborative inference via ensembles on the edge'. Together they form a unique fingerprint.

Cite this