Joint Privacy Enhancement and Quantization in Federated Learning

Natalie Lang, Elad Sofer, Tomer Shaked, Nir Shlezinger

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Federated learning (FL) is an emerging paradigm for training machine learning models using possibly private data available at edge devices. The distributed operation of FL gives rise to challenges that are not encountered in centralized machine learning, including the need to preserve the privacy of the local datasets, and the communication load due to the repeated exchange of updated models. These challenges are often tackled individually via techniques that induce some distortion on the updated models, e.g., local differential privacy (LDP) mechanisms and lossy compression. In this work we propose a method coined joint privacy enhancement and quantization (JoPEQ), which jointly implements lossy compression and privacy enhancement in FL settings. In particular, JoPEQ utilizes vector quantization based on random lattice, a universal compression technique whose byproduct distortion is statistically equivalent to additive noise. This distortion is leveraged to enhance privacy by augmenting the model updates with dedicated multivariate privacy preserving noise. We show that JoPEQ simultaneously quantizes data according to a required bit-rate while holding a desired privacy level, without notably affecting the utility of the learned model. This is shown via analytical LDP guarantees, distortion and convergence bounds derivation, and numerical studies. Finally, we empirically assert that JoPEQ demolishes common attacks known to exploit privacy leakage.

Original languageEnglish
Pages (from-to)295-310
Number of pages16
JournalIEEE Transactions on Signal Processing
Volume71
DOIs
StatePublished - 1 Jan 2023

Keywords

  • Federated learning
  • differential privacy
  • quantization

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Joint Privacy Enhancement and Quantization in Federated Learning'. Together they form a unique fingerprint.

Cite this