LeanConvNets: Low-Cost Yet Effective Convolutional Neural Networks

Jonathan Ephrath, Moshe Eliasof, Lars Ruthotto, Eldad Haber, Eran Treister

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

Convolutional Neural Networks (CNNs) have become indispensable for solving machine learning tasks in speech recognition, computer vision, and other areas that involve high-dimensional data. A CNN filters the input feature using a network containing spatial convolution operators with compactly supported stencils. In practice, the input data and the hidden features consist of a large number of channels, which in most CNNs are fully coupled by the convolution operators. This coupling leads to immense computational cost in the training and prediction phase. In this article, we introduce LeanConvNets that are derived by sparsifying fully-coupled operators in existing CNNs. Our goal is to improve the efficiency of CNNs by reducing the number of weights, floating point operations and latency times, with minimal loss of accuracy. Our lean convolution operators involve tuning parameters that controls the trade-off between the network's accuracy and computational costs. These convolutions can be used in a wide range of existing networks, and we exemplify their use in residual networks (ResNets). Using a range of benchmark problems from image classification and semantic segmentation, we demonstrate that the resulting LeanConvNet's accuracy is close to state-of-the-art networks while being computationally less expensive. In our tests, the lean versions of ResNet in most cases outperform comparable reduced architectures such as MobileNets and ShuffleNets.

Original languageEnglish
Title of host publication36th International Conference on Machine Learning Workshop (ICML), Long Beach, CA, USA, 2019
Pages894-904
Number of pages11
Volume14
Edition4
DOIs
StatePublished - 1 May 2020
Externally publishedYes

Publication series

NameIEEE Journal on Selected Topics in Signal Processing
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISSN (Print)1932-4553

Keywords

  • Moshe: Computer vision
  • deep convolutional neural networks
  • intelligent systems
  • machine learning

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'LeanConvNets: Low-Cost Yet Effective Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this