ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Projection spectral analysis: A unified approach to PCA and ICA with incremental learning

  • Kang, Hoon (School of Electrical and Electronics Engineering, Chung-Ang University) ;
  • Lee, Hyun Su (School of Electrical and Electronics Engineering, Chung-Ang University)
  • Received : 2017.03.04
  • Accepted : 2018.05.03
  • Published : 2018.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

Projection spectral analysis is investigated and refined in this paper, in order to unify principal component analysis and independent component analysis. Singular value decomposition and spectral theorems are applied to nonsymmetric correlation or covariance matrices with multiplicities or singularities, where projections and nilpotents are obtained. Therefore, the suggested approach not only utilizes a sum-product of orthogonal projection operators and real distinct eigenvalues for squared singular values, but also reduces the dimension of correlation or covariance if there are multiple zero eigenvalues. Moreover, incremental learning strategies of projection spectral analysis are also suggested to improve the performance.

Keywords

References

  1. H. Kang, Projection spectral analysis, Int. J. Control, Autom. Syst. 13 (2015), no. 6, 1530-1537. https://doi.org/10.1007/s12555-014-0286-y
  2. A. W. Naylor and G. R. Sell, Linear operator theory in engineering and science - Applied mathematical sciences, vol. 40 Springer-Verlag, Inc., New York, 1982.
  3. D. Hebb, Organization of behavior, Science Edition, Inc., New York, 1961.
  4. J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl Acad. Sci. USA, Biophys. 79 (1982), 2554-2558. https://doi.org/10.1073/pnas.79.8.2554
  5. B. Kosko, Bidirectional associative memories, IEEE Trans. System, Man, Cybern. 18 (1988), no. 1, 49-60. https://doi.org/10.1109/21.87054
  6. M. Turk and A. Pentland, Eigenfaces for recognition, J. Cogn. Neurosci. 3 (1991), no. 1, 71-86. https://doi.org/10.1162/jocn.1991.3.1.71
  7. A. Hyvarinen, J. Karhunen, and E. Oja, Independent component analysis, John Wiley and Sons, Inc., New York, 2001.
  8. J. V. Stone, Independent component analysis - A tutorial introduction, The MIT Press, Cambridge, MA, 2004.
  9. A. Hyvarinen, Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans. Neural Netw. 10 (1999), no. 3, 626-634. https://doi.org/10.1109/72.761722
  10. H. Kang, Multilayered associative neural networks (m. a. n. n.): Storage capacity vs. perfect recall, IEEE Trans. Neural Networks 5 (1994), no. 5, 812-822. https://doi.org/10.1109/72.317732
  11. G. E. Hinton, S. Osindero, and Y. W. Teh, A fast learning algorithm for deep belief nets, Neural Comput. 18 (2006), no. 7 1527-1554. https://doi.org/10.1162/neco.2006.18.7.1527
  12. Y. Bengio, Learning deep architecture for AI, Found. Trends Mach. Learn. 2 (2009), no. 1, 1-127. https://doi.org/10.1561/2200000006
  13. Y. LeCunn, et al., Gradient-based learning applied to document recognition, Proc. IEEE 86 (1998), 1-46.
  14. Y. LeCunn, K. Kavukcuoglu, and C. Farabet, Convolutional networks and applications in vision, Proc. IEEE Int. Symp. Circuits Syst., Paris, France, 2010, pp. 253-256.
  15. H. Kang, Associative cubes in unsupervised learning for robust gray-scale image recognition, Proc. 3rd Int. Symposium on Neural Networks, Advances in Neural Networks - ISNN 2006, Springer-Verlag, Berlin Heidelberg (J. Wang et al., ed.), vol. LNCS 3972, (2006), pp. 581-588.
  16. C.-T. Chen, Linear system theory and design, Oxford University Press, Inc., New York, 1999.
  17. S. Shimizu, et al., A linear non-Gaussian acyclic model for causal discovery, J. Mach. Learn. Res. 7 (2006), 2003-2020.
  18. V. Calhoun, et al., A method for making group inferences from functional MRI data using independent component analysis, Hum. Brain Mapp. 14 (2001), 140-151. https://doi.org/10.1002/hbm.1048
  19. M. Gutmann and A. Hyvarinen, Noise-constrastive estimation: A new estimation principle for unnormalized statistical models, Proc. Int. Conf. Artif. Intell. Statistics Sardinia, Italy, May 13-15, 2010, pp. 297-304.