Over-the-Air Federated Learning from Heterogeneous Data

Tomer Sery, Nir Shlezinger, Kobi Cohen, Yonina Eldar

Research output: Contribution to journalArticlepeer-review

34 Scopus citations

Abstract

We focus on over-the-air (OTA) Federated Learning (FL), which has been suggested recently to reduce the communication overhead of FL due to the repeated transmissions of the model updates by a large number of users over the wireless channel. In OTA FL, all users simultaneously transmit their updates as analog signals over a multiple access channel, and the server receives a superposition of the analog transmitted signals. However, this approach results in the channel noise directly affecting the optimization procedure, which may degrade the accuracy of the trained model. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local stochastic gradient descent (SGD) FL algorithm, introducing precoding at the users and scaling at the server, which gradually mitigates the effect of noise. We analyze the convergence of COTAF to the loss minimizing model and quantify the effect of a statistically heterogeneous setup, i.e. when the training data of each user obeys a different distribution. Our analysis reveals the ability of COTAF to achieve a convergence rate similar to that achievable over error-free channels. Our simulations demonstrate the improved convergence of COTAF over vanilla OTA local SGD for training using non-synthetic datasets. Furthermore, we numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.

Original languageEnglish
Article number9459539
Pages (from-to)3796-3811
Number of pages16
JournalIEEE Transactions on Signal Processing
Volume69
DOIs
StatePublished - 1 Jan 2021

Keywords

  • Machine learning
  • gradient methods
  • optimization
  • wireless communication

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Over-the-Air Federated Learning from Heterogeneous Data'. Together they form a unique fingerprint.

Cite this