Amended cross-entropy cost: An approach for encouraging diversity in classification ensemble

Ron Shoham, Haim Permuter

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

In the field of machine learning, the training of an ensemble of models is a very common method for reducing the variance of the prediction, and yields better results. Many researches indicate that diversity between the predictions of the models is important for the ensemble performance. However, for Deep Learning classification tasks there is no explicit way to encourage diversity. Negative Correlation Learning (NCL) is a method for doing so in regression tasks. In this work we develop a novel algorithm inspired by NCL to explicitly encourage diversity in Deep Neural Networks (DNNs) for classification. In the development of the algorithm we first assume that the same training characteristics that hold in NCL must also hold when training an ensemble for classification. We also suggest the Stacked Diversified Mixture of Classifiers (SDMC), which is based on our outcome. SDMC is a layer that aims to replace the final layer of a DNN classifier. It can be easily applied on any model, while the cost in terms of number of parameters and computational power is relatively low.

Original languageEnglish
Title of host publicationCyber Security Cryptography and Machine Learning - 3rd International Symposium, CSCML 2019, Proceedings
EditorsShlomi Dolev, Danny Hendler, Sachin Lodha, Moti Yung
PublisherSpringer Verlag
Pages202-207
Number of pages6
ISBN (Print)9783030209506
DOIs
StatePublished - 1 Jan 2019
Event3rd International Symposium on Cyber Security Cryptography and Machine Learning, CSCML 2019 - Beer Sheva, Israel
Duration: 27 Jun 201928 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11527 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd International Symposium on Cyber Security Cryptography and Machine Learning, CSCML 2019
Country/TerritoryIsrael
CityBeer Sheva
Period27/06/1928/06/19

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science (all)

Fingerprint

Dive into the research topics of 'Amended cross-entropy cost: An approach for encouraging diversity in classification ensemble'. Together they form a unique fingerprint.

Cite this