TY - GEN
T1 - Amended cross-entropy cost
T2 - 3rd International Symposium on Cyber Security Cryptography and Machine Learning, CSCML 2019
AU - Shoham, Ron
AU - Permuter, Haim
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2019.
PY - 2019/5/19
Y1 - 2019/5/19
N2 - In the field of machine learning, the training of an ensemble of models is a very common method for reducing the variance of the prediction, and yields better results. Many researches indicate that diversity between the predictions of the models is important for the ensemble performance. However, for Deep Learning classification tasks there is no explicit way to encourage diversity. Negative Correlation Learning (NCL) is a method for doing so in regression tasks. In this work we develop a novel algorithm inspired by NCL to explicitly encourage diversity in Deep Neural Networks (DNNs) for classification. In the development of the algorithm we first assume that the same training characteristics that hold in NCL must also hold when training an ensemble for classification. We also suggest the Stacked Diversified Mixture of Classifiers (SDMC), which is based on our outcome. SDMC is a layer that aims to replace the final layer of a DNN classifier. It can be easily applied on any model, while the cost in terms of number of parameters and computational power is relatively low.
AB - In the field of machine learning, the training of an ensemble of models is a very common method for reducing the variance of the prediction, and yields better results. Many researches indicate that diversity between the predictions of the models is important for the ensemble performance. However, for Deep Learning classification tasks there is no explicit way to encourage diversity. Negative Correlation Learning (NCL) is a method for doing so in regression tasks. In this work we develop a novel algorithm inspired by NCL to explicitly encourage diversity in Deep Neural Networks (DNNs) for classification. In the development of the algorithm we first assume that the same training characteristics that hold in NCL must also hold when training an ensemble for classification. We also suggest the Stacked Diversified Mixture of Classifiers (SDMC), which is based on our outcome. SDMC is a layer that aims to replace the final layer of a DNN classifier. It can be easily applied on any model, while the cost in terms of number of parameters and computational power is relatively low.
UR - http://www.scopus.com/inward/record.url?scp=85068207287&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-20951-3_18
DO - 10.1007/978-3-030-20951-3_18
M3 - Conference contribution
AN - SCOPUS:85068207287
SN - 9783030209506
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 202
EP - 207
BT - Cyber Security Cryptography and Machine Learning - 3rd International Symposium, CSCML 2019
A2 - Dolev, Shlomi
A2 - Hendler, Danny
A2 - Lodha, Sachin
A2 - Yung, Moti
PB - Springer Verlag
Y2 - 27 June 2019 through 28 June 2019
ER -