Parallel Implementations of the Self-Organizing Network for Normal Mixtures Lee, Chul-Hee; Ahn, Sung-Mahn;
This article proposes a couple of parallel implementations of the self-organizing network for normal mixtures. In principle, self-organizing networks should be able to be implemented in a parallel computing environment without issue. However, the network for normal mixtures has inherent problem in being operated parallel in pure sense due to estimating conditional expectations of the mixing proportion in each iteration. This article shows the result of the parallel implementations of the network using Java. According to the results, both of the implementations achieved a faster execution without any performance degradation.
Ahn, S. and Kim, S. W. (2011). A self-organizing network for normal mixtures, The Korean Communications in Statistics, 18, 837-849.
Angel, L. D. (1989). The Technology of Parallel Processing, Prentice-Hall.
Celeux, G., Chretien, S., Forbes, F. and Mkhadri, A. (2001). A component-wise EM algorithm for mixtures, Journal of Computational and Graphical Statistics, 10, 697-712.
Dempster, A., Laird, N. and Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B, 39, 1-38.
Kuhn, R. H. and Padua, D. A. (1981). Tutorial on parallel processing, IEEE Tenth International Conference on Parallel Processing.
Liu, C. and Sun, D. X. (1997). Acceleration of EM algorithm for mixture models using ECME, Proceedings of the Statistical Computing Section, Alexandria, VA: ASA, 109-114.
Meng, X. L. and Rubin, D. B. (1991). Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm, Journal of the American Statistical Association, 86, 899-909.
Molodovan, D. I. (1986). Modern Parallel processing, University of Southern California.
Neal, R. M. and Hinton, G. E. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants, Learning in Graphical Model, 355-368 (Jordan, M. I. et al, Eds). Kluwer Academic Press, Norwell, MA.
Taylor, S. (1989). Parallel Logic Programming Techniques, Prentice-Hall.
Tian, T. and Shih, C-P. (2009). Software techniques for Shared-Cache Multi-Core Systems, Intel Software Network, http://software.intel.com/ en-us/articles/software-techniques-for-shared-cache-multicore-systems/
Titterington, D. M. (1984). Recursive parameter estimation using incomplete data, Journal of the Royal Statistical Society, Series B, 46, 257-267.
Xu, L. and Jordan, M. I. (1996). On convergence properties of the EM algorithm for Gaussian mixtures, Neural Computation, 8, 129-151.
Yin, H. and Allinson, N. (2001). Self-organizing mixture networks for probability density estimation, IEEE Transactions on Neural Network, 12, 405-411.