TY - GEN
T1 - Input-dependably feature-map pruning
AU - Waissman, Atalya
AU - Bar-Hillel, Aharon
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2018.
PY - 2018/1/1
Y1 - 2018/1/1
N2 - Deep neural networks are an accurate tool for solving, among other things, vision tasks. The computational cost of these networks is often high, preventing their adoption in many real time applications. Thus, there is a constant need for computational saving in this research domain. In this paper we suggest trading accuracy with computation using a gated version of Convolutional Neural Networks (CNN). The gated network selectively activates only a portion of its feature-maps, depending on the given example to be classified. The network’s ‘gates’ imply which feature-maps are necessary for the task, and which are not. Specifically, full feature maps are considered for omission, to enable computational savings in a manner compliant with GPU hardware constraints. The network is trained using a combination of back-propagation for standard weights, minimizing an error-related loss, and reinforcement learning for the gates, minimizing a loss related to the number of feature maps used. We trained and evaluated a gated version of dense-net on the CIFAR-10 dataset [1]. Our results show that with slight impact on the network accuracy, a potential acceleration of up to ×3 might be obtained.
AB - Deep neural networks are an accurate tool for solving, among other things, vision tasks. The computational cost of these networks is often high, preventing their adoption in many real time applications. Thus, there is a constant need for computational saving in this research domain. In this paper we suggest trading accuracy with computation using a gated version of Convolutional Neural Networks (CNN). The gated network selectively activates only a portion of its feature-maps, depending on the given example to be classified. The network’s ‘gates’ imply which feature-maps are necessary for the task, and which are not. Specifically, full feature maps are considered for omission, to enable computational savings in a manner compliant with GPU hardware constraints. The network is trained using a combination of back-propagation for standard weights, minimizing an error-related loss, and reinforcement learning for the gates, minimizing a loss related to the number of feature maps used. We trained and evaluated a gated version of dense-net on the CIFAR-10 dataset [1]. Our results show that with slight impact on the network accuracy, a potential acceleration of up to ×3 might be obtained.
KW - Acceleration
KW - Conditional computation
KW - Feature-map
KW - Neural networks
KW - Pruning
UR - http://www.scopus.com/inward/record.url?scp=85054821842&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01418-6_69
DO - 10.1007/978-3-030-01418-6_69
M3 - Conference contribution
AN - SCOPUS:85054821842
SN - 9783030014179
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 706
EP - 713
BT - Artificial Neural Networks and Machine Learning – ICANN 2018 - 27th International Conference on Artificial Neural Networks, 2018, Proceedings
A2 - Kurkova, Vera
A2 - Hammer, Barbara
A2 - Manolopoulos, Yannis
A2 - Iliadis, Lazaros
A2 - Maglogiannis, Ilias
PB - Springer Verlag
T2 - 27th International Conference on Artificial Neural Networks, ICANN 2018
Y2 - 4 October 2018 through 7 October 2018
ER -