An EM Algorithm for a Doubly Smoothed MLE in Normal Mixture Models Seo, Byung-Tae;
It is well known that the maximum likelihood estimator(MLE) in normal mixture models with unequal variances does not fall in the interior of the parameter space. Recently, a doubly smoothed maximum likelihood estimator(DS-MLE) (Seo and Lindsay, 2010) was proposed as a general alternative to the ordinary maximum likelihood estimator. Although this method gives a natural modification to the ordinary MLE, its computation is cumbersome due to intractable integrations. In this paper, we derive an EM algorithm for the DS-MLE under normal mixture models and propose a fast computational tool using a local quadratic approximation. The accuracy and speed of the proposed method is then presented via some numerical studies.
EM algorithm;normal mixture;doubly-smoothed MLE;quadratic approximation;
Chen, J., Tan, X. and Zhang, R. (2008). Consistency of penalized mle for normal mixtures in mean and variance, Statistica Sinica, 18, 443-465.
Ciuperca, G. A., Ridolfi, A. and Idier, J. (2003). Penalized maximum likelihood estimator for normal mixtures, Scandinavian Journal of Statistics, 30, 45-59.
Crawford, S. K., Degroot, M. H., Kadane, J. B. and Small, M. J. (1992). Modeling lake chemistry distributions: Approximate Bayesian methods for estimating a finite mixture model, Technometrics, 34, 441-453.
Fraley, C. and Raftery, A. E. (2007). Bayesian regularization for normal mixture estimation and model-based clustering, Journal of Classification, 24, 155-181.
Hathaway, R. J. (1985). A constrained formulation of maximum likelihood estimation for normal mixture distributions, Annals of Statistics, 13, 795-800.
Hathaway, R. J. (1986). A constrained EM-algorithm for univariate normal mixtures, Computational Statistics & Data Analysis, 23, 211-230.
Ingrassia, S. and Rocci, R. (2007). Constrained monotone EM algorithms for finite mixture of multi- variate Gaussians, Computational Statistics & Data Analysis, 51, 5339-5351.
Kiefer, J. andWolfowitz, J. (1956). Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters, Annals of Statistics, 27, 886-906.
Lindsay, B. G., Markatou, M., Ray, S., Yang, K. and Chen, S. (2008). Quadratic distances on probabilities: A unified foundation, Annals of Statistics, 36, 983-1006.
Ray, S. and Lindsay, B. G. (2008). Model selection in high-dimensions: A quadratic-risk based approach, Journal of the Royal Statistical Society, Series B, 70, 95-118.
Seo, B. and Lindsay, B. G. (2010). A computational strategy for doubly smoothed MLE exemplified in the normal mixture model, Computational Statistics and Data Analysis, 54, 1930-1941.
Seo, B. and Lindsay, B. G. (2011). A universally consistent modification of maximum likelihood, Submitted.
Tanaka, K. and Takemura, A. (2006). Strong consistency of the maximum likelihood estimator for finite mixtures of location-scale distributions when the scale parameters are exponentially small, Bernoulli, 12, 1003-1017.