Neural Network Representation Control: Gaussian Isolation Machines and CVC Regularization.

In many cases, neural network classifiers are likely
to be exposed to input data that is outside of their training
distribution data. Samples from outside the distribution may
be classified as an existing class with high probability by
softmax-based classifiers; such incorrect classifications affect the
performance of the classifiers and the applications/systems that
depend on them. Previous research aimed at distinguishing training distribution data from out-of-distribution data (OOD) has
proposed detectors that are external to the classification method.
We present Gaussian isolation machine (GIM), a novel hybrid
(generative-discriminative) classifier aimed at solving the problem
arising when OOD data is encountered. The GIM is based on a
neural network and utilizes a new loss function that imposes a
distribution on each of the trained classes in the neural network’s
output space, which can be approximated by a Gaussian. The
proposed GIM’s novelty lies in its discriminative performance
and generative capabilities, a combination of characteristics not
usually seen in a single classifier. The GIM achieves state-ofthe-art classification results on image recognition and sentiment
analysis benchmarking datasets and can also deal with OOD
inputs