A Fast EM Algorithm for Gaussian Mixtures Jung, Hye-Kyung; Seo, Byung-Tae;
The EM algorithm is the most important tool to obtain the maximum likelihood estimator in finite mixture models due to its stability and simplicity. However, its convergence rate is often slow because the conventional EM algorithm is based on a large missing data space. Several techniques have been proposed in the literature to reduce the missing data space. In this paper, we review existing methods and propose a new EM algorithm for Gaussian mixtures, which reduces the missing data space while preserving the stability of the conventional EM algorithm. The performance of the proposed method is evaluated with other existing methods via simulation studies.
EM algorithm;ECM algorithm;constrained Newton method;
Biernacki, C. and Chretien, S. (2003). Degeneracy in the maximum likelihood estimation of univariate gaussian mixtures with EM, Statistics and Probability Letters, 61, 373-382.
Celeux, G., Chretien, S., Forbes, F. and Mkhadri, A. (1999). A component-wise EM algorithm for mixtures, Technical Report, Inria 3746, (http://www.inria.fr/RRRT/publications-fra.html).
Celeux, G., Chretien, S., Forbes, F. and Mkhadri, A. (2001). A component-wise EM algorithm for mixtures, Journal of Computational and Graphical Statistics, 10, 697-712.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood for incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B, 39, 1-38.
Fessler, J. A. and Hero, A. O. (1994). Space-alternating generalized expectation-maximization algorithm, IEEE Transactions on Signal Processing, 42, 2664-2677.
Liu, C. and Rubin, D. B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence, Biometrika, 81, 633-648.
Liu, C. and Sun, D. X. (1997). Acceleration of EM algorithm for mixture models using ECME, Proceedings of the Statistical Computing Section, Alexandria, VA: ASA,, 109-114.
McLachlan, G. J. and Peel, D. (2000). Finite Mixture Models, Hohn Wiley and Sons Ltd., New York.
Meng, X.-L. and Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework, Biometrika, 80, 267-278.
Pilla, R. S. and Lindsay, B. G. (1996). Alternative EM methods for nonparametric finite mixture models, Biometrika, 88, 535-550.
Wang, Y. (2007). On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution, Journal of the Royal Statistical Society, Series B, 69, 185-198.