A Comparative Experiment on Dimensional Reduction Methods Applicable for Dissimilarity-Based Classifications

Title & Authors
A Comparative Experiment on Dimensional Reduction Methods Applicable for Dissimilarity-Based Classifications
Kim, Sang-Woon;

Abstract
This paper presents an empirical evaluation on dimensionality reduction strategies by which dissimilarity-based classifications (DBC) can be implemented efficiently. In DBC, classification is not based on feature measurements of individual objects (a set of attributes), but rather on a suitable dissimilarity measure among the individual objects (pair-wise object comparisons). One problem of DBC is the high dimensionality of the dissimilarity space when a lots of objects are treated. To address this issue, two kinds of solutions have been proposed in the literature: prototype selection (PS)-based methods and dimension reduction (DR)-based methods. In this paper, instead of utilizing the PS-based or DR-based methods, a way of performing DBC in Eigen spaces (ES) is considered and empirically compared. In ES-based DBC, classifications are performed as follows: first, a set of principal eigenvectors is extracted from the training data set using a principal component analysis; second, an Eigen space is expanded using a subset of the extracted and selected Eigen vectors; third, after measuring distances among the projected objects in the Eigen space using $\small{l_p}$-norms as the dissimilarity, classification is performed. The experimental results, which are obtained using the nearest neighbor rule with artificial and real-life benchmark data sets, demonstrate that when the dimensionality of the Eigen spaces has been selected appropriately, compared to the PS-based and DR-based methods, the performance of the ES-based DBC can be improved in terms of the classification accuracy.
Keywords
dissimilarity-based classification(DBC);prototype selection based methods;dimension reduction based methods;Eigen space based methods;
Language
Korean
Cited by
References
1.
S. -W. Kim and R. P. W. Duin, "On optimizing dissimilarity-based classifier using multi-level fusion strategies (in Korean)," Journal of the Institute of Electronics Engineers of Korea, vol. 45, no. CI-5, pp. 15-24, 2008.

2.
R. P. W. Duin, "The dissimilarity representation for finding universals from particulars by an anti-essentialist approach," Pattern Recognition Letters, vol. 64, pp. 37-43, 2015.

3.
J. Laub, V. Roth, J. M. Buhmann, and K.-R. Muller, "On the information and representation of non-Euclidean pairwise data," Pattern Recognition, vol. 39, pp. 1815-1826, 2006.

4.
S. -W. Kim and S. -H. Kim, "On optimizing dissimilarity-based classifications using a DTW and fusion strategies (in Korean)," Journal of the Institute of Electronics Engineers of Korea, vol. 47, no. CI-2, pp. 212-219, 2010.

5.
E. Pekalska, R. P. W. Duin, and P. Paclik, "Prototype selection for dissimilarity-based classifiers," Pattern Recognition, vol. 39, pp. 189-208, 2006.

6.
K. Riesen, V. Kilchherr, and H. Bunke, "Reducing the dimensionality of vector space embeddings of graphs," in Proc. of 5th Int'l Conf. on Machine Learning and Data Mining, vol. LNAI-4571 pp. 563-573, 2007.

7.
S. -W. Kim, "An empirical evaluation on dimensionality reduction schemes for dissimilarity -based classifications," Pattern Recognition Letters, vol. 32, pp. 816-823, 2011.

8.
S. -W. Kim, "Dissimilarity representations using lp-norms in Eigen spaces," in Proc. of the 2015 Int'l Conf. on Image Processing, Computer Vision, Pattern Recognition, Las Vegas, Nevada, CSREA Press, pp. 242-248, 2015.

9.
D. He and J. Wang, "Parallel computing of eigenvalue of doubly stochastic matrix," in Proc. of the 5th Int'l Conf. on Algorithms and Architecture for Parallel Processing, Beijing, China, pp. 355-358, 2002.

10.
R. P. W. Duin, M. Bicego, M. Orozco-Alzate, S. -W. Kim, and M. Loog, "Metric learning in dissimilarity space for improved nearest neighbor performance," in Proc. of S+SSPR2014, Joensuu, Finland, LNCS8621, pp. 183-192, 2014.

11.
N. Kwak, "Principal component analysis based on l1-norm maximization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 9, pp. 1672-1680, 2008.