Improve Robustness of Deep Neural Networks by Coding

Kunping Huang; Netanel Raviv; Siddharth Jain; Pulakesh Upadhyaya; Jehoshua Bruck; Paul H. Siegel; Anxiao Andrew Jiang

doi:10.1109/ITA50056.2020.9244998

Improve Robustness of Deep Neural Networks by Coding

Kunping Huang, Netanel Raviv, Siddharth Jain, Pulakesh Upadhyaya, Jehoshua Bruck, Paul H. Siegel, Anxiao Andrew Jiang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Deep neural networks (DNNs) typically have many weights. When errors appear in their weights, which are usually stored in non-volatile memories, their performance can degrade significantly. We review two recently presented approaches that improve the robustness of DNNs in complementary ways. In the first approach, we use error-correcting codes as external redundancy to protect the weights from errors. A deep reinforcement learning algorithm is used to optimize the redundancy-performance tradeoff. In the second approach, internal redundancy is added to neurons via coding. It enables neurons to perform robust inference in noisy environments.

Original language	English
Title of host publication	2020 Information Theory and Applications Workshop, ITA 2020
Publisher	Institute of Electrical and Electronics Engineers
ISBN (Electronic)	9781728141909
DOIs	https://doi.org/10.1109/ITA50056.2020.9244998
State	Published - 2 Feb 2020
Externally published	Yes
Event	2020 Information Theory and Applications Workshop, ITA 2020 - San Diego, United States Duration: 2 Feb 2020 → 7 Feb 2020

Publication series

Name	2020 Information Theory and Applications Workshop, ITA 2020

Conference

Conference	2020 Information Theory and Applications Workshop, ITA 2020
Country/Territory	United States
City	San Diego
Period	2/02/20 → 7/02/20

ASJC Scopus subject areas

Artificial Intelligence
Computational Theory and Mathematics
Computer Science Applications
Information Systems and Management
Control and Optimization

Access to Document

10.1109/ITA50056.2020.9244998

Cite this

Huang, K., Raviv, N., Jain, S., Upadhyaya, P., Bruck, J., Siegel, P. H., & Jiang, A. A. (2020). Improve Robustness of Deep Neural Networks by Coding. In 2020 Information Theory and Applications Workshop, ITA 2020 Article 9244998 (2020 Information Theory and Applications Workshop, ITA 2020). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ITA50056.2020.9244998

@inproceedings{b01c92b3cd1a4ea8bd21968534a472e2,

title = "Improve Robustness of Deep Neural Networks by Coding",

abstract = "Deep neural networks (DNNs) typically have many weights. When errors appear in their weights, which are usually stored in non-volatile memories, their performance can degrade significantly. We review two recently presented approaches that improve the robustness of DNNs in complementary ways. In the first approach, we use error-correcting codes as external redundancy to protect the weights from errors. A deep reinforcement learning algorithm is used to optimize the redundancy-performance tradeoff. In the second approach, internal redundancy is added to neurons via coding. It enables neurons to perform robust inference in noisy environments.",

author = "Kunping Huang and Netanel Raviv and Siddharth Jain and Pulakesh Upadhyaya and Jehoshua Bruck and Siegel, {Paul H.} and Jiang, {Anxiao Andrew}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 Information Theory and Applications Workshop, ITA 2020 ; Conference date: 02-02-2020 Through 07-02-2020",

year = "2020",

month = feb,

day = "2",

doi = "10.1109/ITA50056.2020.9244998",

language = "English",

series = "2020 Information Theory and Applications Workshop, ITA 2020",

publisher = "Institute of Electrical and Electronics Engineers",

booktitle = "2020 Information Theory and Applications Workshop, ITA 2020",

address = "United States",

}

Huang, K, Raviv, N, Jain, S, Upadhyaya, P, Bruck, J, Siegel, PH & Jiang, AA 2020, Improve Robustness of Deep Neural Networks by Coding. in 2020 Information Theory and Applications Workshop, ITA 2020., 9244998, 2020 Information Theory and Applications Workshop, ITA 2020, Institute of Electrical and Electronics Engineers, 2020 Information Theory and Applications Workshop, ITA 2020, San Diego, United States, 2/02/20. https://doi.org/10.1109/ITA50056.2020.9244998

Improve Robustness of Deep Neural Networks by Coding. / Huang, Kunping; Raviv, Netanel; Jain, Siddharth et al.
2020 Information Theory and Applications Workshop, ITA 2020. Institute of Electrical and Electronics Engineers, 2020. 9244998 (2020 Information Theory and Applications Workshop, ITA 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improve Robustness of Deep Neural Networks by Coding

AU - Huang, Kunping

AU - Raviv, Netanel

AU - Jain, Siddharth

AU - Upadhyaya, Pulakesh

AU - Bruck, Jehoshua

AU - Siegel, Paul H.

AU - Jiang, Anxiao Andrew

PY - 2020/2/2

Y1 - 2020/2/2

N2 - Deep neural networks (DNNs) typically have many weights. When errors appear in their weights, which are usually stored in non-volatile memories, their performance can degrade significantly. We review two recently presented approaches that improve the robustness of DNNs in complementary ways. In the first approach, we use error-correcting codes as external redundancy to protect the weights from errors. A deep reinforcement learning algorithm is used to optimize the redundancy-performance tradeoff. In the second approach, internal redundancy is added to neurons via coding. It enables neurons to perform robust inference in noisy environments.

AB - Deep neural networks (DNNs) typically have many weights. When errors appear in their weights, which are usually stored in non-volatile memories, their performance can degrade significantly. We review two recently presented approaches that improve the robustness of DNNs in complementary ways. In the first approach, we use error-correcting codes as external redundancy to protect the weights from errors. A deep reinforcement learning algorithm is used to optimize the redundancy-performance tradeoff. In the second approach, internal redundancy is added to neurons via coding. It enables neurons to perform robust inference in noisy environments.

UR - http://www.scopus.com/inward/record.url?scp=85097331968&partnerID=8YFLogxK

U2 - 10.1109/ITA50056.2020.9244998

DO - 10.1109/ITA50056.2020.9244998

M3 - Conference contribution

AN - SCOPUS:85097331968

T3 - 2020 Information Theory and Applications Workshop, ITA 2020

BT - 2020 Information Theory and Applications Workshop, ITA 2020

PB - Institute of Electrical and Electronics Engineers

T2 - 2020 Information Theory and Applications Workshop, ITA 2020

Y2 - 2 February 2020 through 7 February 2020

ER -

Improve Robustness of Deep Neural Networks by Coding

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this