TY - JOUR
T1 - Regularized mixture discriminant analysis
AU - Halbe, Zohar
AU - Aladjem, Mayer
N1 - Funding Information:
The authors wish to thank associated editor Prof. Robert P.W. Duin and the reviewers for their critical reading of the manuscript and helpful comments. This work was supported in part by the Paul Ivanier Center for Robotics and Production Management, Ben-Gurion University of the Negev, Israel.
PY - 2007/11/1
Y1 - 2007/11/1
N2 - In this paper, we seek a Gaussian mixture model (GMM) of the class-conditional densities for plug-in Bayes classification. We propose a method for setting the number of the components and the covariance matrices of the class-conditional GMMs. It compromises between simplicity of the model selection based on the Bayesian information criterion (BIC) and the high accuracy of the model selection based on the cross-validation (CV) estimate of the correct classification rate. We apply an idea of Friedman [Friedman, J.H. 1989. Regularized discriminant analysis. J. Amer. Statist. Assoc., 84, 165-175] to shrink a predefined covariance matrix to a parameterization with substantially reduced degrees of freedom (reduced number of the adjustable parameters). Our method differs from the original Friedman's method by the meaning of the shrinkage. We operate on matrices computed for a certain class while the Friedman's method shrinks matrices from different classes. We compare our method with the conventional methods for setting the GMMs based on the BIC and CV. The experimental results show that our method has the potential to produce parameterizations of the covariance matrices of the GMMs which are better than the parameterizations used in other methods. We observed significant enlargement of the correct classification rates for our method with respect to the other methods which is more pronounced as the training sample size decreases. The latter implies that our method could be an attractive choice for applications based on a small number of training observations.
AB - In this paper, we seek a Gaussian mixture model (GMM) of the class-conditional densities for plug-in Bayes classification. We propose a method for setting the number of the components and the covariance matrices of the class-conditional GMMs. It compromises between simplicity of the model selection based on the Bayesian information criterion (BIC) and the high accuracy of the model selection based on the cross-validation (CV) estimate of the correct classification rate. We apply an idea of Friedman [Friedman, J.H. 1989. Regularized discriminant analysis. J. Amer. Statist. Assoc., 84, 165-175] to shrink a predefined covariance matrix to a parameterization with substantially reduced degrees of freedom (reduced number of the adjustable parameters). Our method differs from the original Friedman's method by the meaning of the shrinkage. We operate on matrices computed for a certain class while the Friedman's method shrinks matrices from different classes. We compare our method with the conventional methods for setting the GMMs based on the BIC and CV. The experimental results show that our method has the potential to produce parameterizations of the covariance matrices of the GMMs which are better than the parameterizations used in other methods. We observed significant enlargement of the correct classification rates for our method with respect to the other methods which is more pronounced as the training sample size decreases. The latter implies that our method could be an attractive choice for applications based on a small number of training observations.
KW - Bayesian information criterion
KW - Classification
KW - Gaussian mixture models
KW - Model selection
KW - Regularized discriminant analysis
UR - http://www.scopus.com/inward/record.url?scp=34548668667&partnerID=8YFLogxK
U2 - 10.1016/j.patrec.2007.06.009
DO - 10.1016/j.patrec.2007.06.009
M3 - Article
AN - SCOPUS:34548668667
SN - 0167-8655
VL - 28
SP - 2104
EP - 2115
JO - Pattern Recognition Letters
JF - Pattern Recognition Letters
IS - 15
ER -