Joint Privacy Enhancement and Quantization in Federated Learning

Federated learning (FL) is an emerging paradigm for training machine learning models using possibly private data available at edge devices. Among the key challenges associated with FL are first the need to preserve the privacy of the local data sets, and second the communication load due to the repeated exchange of updated models; both are often tackled individually with methods whose operation distorts the updated models, e.g., local differential privacy (LDP) mechanisms and lossy compres- sion, respectively. In this work we propose a method for joint privacy enhancement and quantization (JoPEQ), unifying lossy compression and privacy enhancement for FL. JoPEQ utilizes universal vector quantization, where distortion is statistically equivalent to additive noise, and augments the compression distortion with dedicated privacy preserving noise to simultaneously achieve compression and a desired privacy level. We numerically demonstrate that JoPEQ reduces the overall distortion compared to individual LDP and compression, which is translated into improved trained models.