Dirichlet Process Mixtures of Linear Mixed Regressions Kyung, Minjung;
We develop a Bayesian clustering procedure based on a Dirichlet process prior with cluster specific random effects. Gibbs sampling of a normal mixture of linear mixed regressions with a Dirichlet process was implemented to calculate posterior probabilities when the number of clusters was unknown. Our approach (unlike its counterparts) provides simultaneous partitioning and parameter estimation with the computation of the classification probabilities. A Monte Carlo study of curve estimation results showed that the model was useful for function estimation. We find that the proposed Dirichlet process mixture model with cluster specific random effects detects clusters sensitively by combining vague edges into different clusters. Examples are given to show how these models perform on real data.
normal mixture;cluster specific random effect;model-based cluster;linear mixed regression;Dirichlet process;
Banfield, J. D. and Raftery, A. E. (1993). Model-based Gaussian and non-Gaussian clustering, Biometrics, 49, 803-821.
Blackwell, D. and MacQueen, J. B. (1973). Ferguson distributions via Polya urn schemes, Annals of Statistics ,1, 353-355.
Booth, J. G., Casella, G. and Hobert, J. P. (2008). Clustering using objective functions and stochastic search, Journal of Royal Statistical Society: Series B (Statistical Methodology), 70, 119-139.
Breslow, N. E. and Clayton, D. G. (1993). Approximate inference in generalized linear mixed models, Journal of the American Statistical Association, 88, 9-25.
Buonaccorsi, J. P. (1996). Measurement error in the response in the general linear model, Journal of the American Statistical Association, 91, 633-642.
Dasgupta, A. and Raftery, A. E. (1998). Detecting features in spatial point processes with clutter via model-bases clustering, Journal of the American Statistical Association, 93, 294-302.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society: Series B (Methodological), 39, 1-38.
Escobar, M. D. and West, M. (1995). Bayesian density estimation and inference using mixtures, Journal of the American Statistical Association, 90, 577-588.
Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems, Annals of Statistics, 1, 209-230.
Fraley, C. and Raftery, A. E. (2002). Model-based clustering, discriminant analysis, and density estimation, Journal of the American Statistical Association, 97, 611-631.
Hurn, M., Justel, A. and Robert, C. P. (2003). Estimating mixtures of regressions, Journal of Computational and Graphical Statistics, 12, 55-79.
Kiefer, N. M. (1978). Discrete parameter variation: Efficient estimation of a switching regression model, Econometrica, 46, 427-434.
Kyung, M., Gill, J. and Casella G. (2010). Estimation in Dirichlet random effects models, Annals of Statistics, 38, 979-1009.
MacEachern, S. N. and Muller, P. (1998). Estimating mixture of Dirichlet process model, Journal of Computational and Graphical Statistics, 7, 223-238.
McLachlan, G. J. and Basford, K. E. (1988). Mixture Models: Inference and Applications to Clustering, Marcel Dekker, New York.
McLachlan, G. J. and Peel, D. (2000). Finite Mixture Models, Wiley, New York.
Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models, Journal of Computational and Graphical Statistics, 9, 249-265
Quandt, R. E. (1958). The estimation of the parameters of a linear regression system obeying two separate regimes, Journal of the American Statistical Association, 53, 873-880.
Quandt, R. E. and Ramsey, J. B. (1978). Estimating mixtures of normal distributions and switching regressions, Journal of the American Statistical Association, 73, 730-738.
Richardson, S. and Green, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components, Journal of the Royal Statistical Society: Series B (Methodological), 59, 731-792.
Tierney, L. and Kadane, J. B. (1986). Accurate approximations for posterior moments and marginal densities, Journal of the American Statistical Association, 81, 82-86.
Wang, N., Lin, X., Gutierrez, R. G. and Carroll, R. J. (1998). Bias analysis and SIMEX approach in generalized linear mixed measurement error models, Journal of the American Statistical Association, 93, 249-261.
Wolfinger, R. and O'Connell, M. (1993). Generalized linear mixed models a pseudo-likelihood approach, Journal of Statistical Computation and Simulation, 48, 233-243.