Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

A Fast EM Algorithm for Gaussian Mixtures

  • Received : 20111000
  • Accepted : 20111100
  • Published : 2012.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

The EM algorithm is the most important tool to obtain the maximum likelihood estimator in finite mixture models due to its stability and simplicity. However, its convergence rate is often slow because the conventional EM algorithm is based on a large missing data space. Several techniques have been proposed in the literature to reduce the missing data space. In this paper, we review existing methods and propose a new EM algorithm for Gaussian mixtures, which reduces the missing data space while preserving the stability of the conventional EM algorithm. The performance of the proposed method is evaluated with other existing methods via simulation studies.

Keywords

References

  1. Biernacki, C. and Chretien, S. (2003). Degeneracy in the maximum likelihood estimation of univariate gaussian mixtures with EM, Statistics and Probability Letters, 61, 373-382. https://doi.org/10.1016/S0167-7152(02)00396-6
  2. Celeux, G., Chretien, S., Forbes, F. and Mkhadri, A. (1999). A component-wise EM algorithm for mixtures, Technical Report, Inria 3746, (http://www.inria.fr/RRRT/publications-fra.html).
  3. Celeux, G., Chretien, S., Forbes, F. and Mkhadri, A. (2001). A component-wise EM algorithm for mixtures, Journal of Computational and Graphical Statistics, 10, 697-712. https://doi.org/10.1198/106186001317243403
  4. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood for incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B, 39, 1-38.
  5. Fessler, J. A. and Hero, A. O. (1994). Space-alternating generalized expectation-maximization algorithm, IEEE Transactions on Signal Processing, 42, 2664-2677. https://doi.org/10.1109/78.324732
  6. Liu, C. and Rubin, D. B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence, Biometrika, 81, 633-648. https://doi.org/10.1093/biomet/81.4.633
  7. Liu, C. and Sun, D. X. (1997). Acceleration of EM algorithm for mixture models using ECME, Proceedings of the Statistical Computing Section, Alexandria, VA: ASA,, 109-114.
  8. McLachlan, G. J. and Peel, D. (2000). Finite Mixture Models, Hohn Wiley and Sons Ltd., New York.
  9. Meng, X.-L. and Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework, Biometrika, 80, 267-278. https://doi.org/10.1093/biomet/80.2.267
  10. Pilla, R. S. and Lindsay, B. G. (1996). Alternative EM methods for nonparametric finite mixture models, Biometrika, 88, 535-550. https://doi.org/10.1093/biomet/88.2.535
  11. Wang, Y. (2007). On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution, Journal of the Royal Statistical Society, Series B, 69, 185-198. https://doi.org/10.1111/j.1467-9868.2007.00583.x