Quantized convolutional neural networks through the lens of partial differential equations.

Ido Ben-Yair, Gil Ben Shalom, Moshe Eliasof, Eran Treister

Research output: Working paper/PreprintPreprint

Abstract

Quantization of Convolutional Neural Networks (CNNs) is a common approach to ease the computational burden involved in the deployment of CNNs, especially on low-resource edge devices. However, fixed-point arithmetic is not natural to the type of computations involved in neural networks. In this work, we explore ways to improve quantized CNNs using PDE-based perspective and analysis. First, we harness the total variation (TV) approach to apply edge-aware smoothing to the feature maps throughout the network. This aims to reduce outliers in the distribution of values and promote piece-wise constant maps, which are more suitable for quantization. Secondly, we consider symmetric and stable variants of common CNNs for image classification, and Graph Convolutional Networks (GCNs) for graph node-classification. We demonstrate through several experiments that the property of forward stability preserves the action of a network under different quantization rates. As a result, stable quantized networks behave similarly to their non-quantized counterparts even though they rely on fewer parameters. We also find that at times, stability even aids in improving accuracy. These properties are of particular interest for sensitive, resource-constrained, low-power or real-time applications like autonomous driving.
Original languageEnglish GB
Volumeabs/2109.00095
StatePublished - 2021

Publication series

NameCoRR

Fingerprint

Dive into the research topics of 'Quantized convolutional neural networks through the lens of partial differential equations.'. Together they form a unique fingerprint.

Cite this