Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지

Fixed size LS-SVM for multiclassification problems of large data sets

  • Received : 2010.03.12
  • Accepted : 2010.04.30
  • Published : 2010.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Multiclassification is typically performed using voting scheme methods based on combining a set of binary classifications. In this paper we use multiclassification method with a hat matrix of least squares support vector machine (LS-SVM), which can be regarded as the revised one-against-all method. To tackle multiclass problems for large data, we use the $Nystr\ddot{o}m$ approximation and the quadratic Renyi entropy with estimation in the primal space such as used in xed size LS-SVM. For the selection of hyperparameters, generalized cross validation techniques are employed. Experimental results are then presented to indicate the performance of the proposed procedure.

Keywords

References

  1. Allwein, E. L., Schapire, R. E. and Singer, Y. (2000). Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1, 113-141.
  2. Dietterich, T. G. and Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2, 263-286.
  3. Espinoza, M., Suykens, J. A. K. and De Moor, B. (2005). Load forecasting using least squares support vector machines. Lecture Notes in Computer Science, 3512, 1018-1026.
  4. Girolami, M. (2003). Orthogonal series density estimation and kernel eigenvalue problem. Neural Computation, 14, 669-688.
  5. Hwang, C. (2007). Kernel machine for Poisson regression. Journal of Korean Data & Information Science Society, 18, 767-772.
  6. Hwang, C. (2008). Mixed effects kernel binomial regression. Journal of Korean Data & Information Science Society, 19, 1327-1334.
  7. Kimeldorf, G. S. and Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and its Applications, 33, 82-95. https://doi.org/10.1016/0022-247X(71)90184-3
  8. Lee, Y. , Lin, Y. and Wahba, G. (2001). Multicategory support vector machines, Technical Report 1043, Department of Statistics, University of Wisconsin.
  9. Mercer, J. (1909). Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, A, 415-446.
  10. Rifkin, R. and Klautau, A. (2004). In defense of one-vs-all classification. Journal of Machine Learning Research, 5, 101-141.
  11. Shim, J., Bae, J. and Hwang, C. (2008). Multiclass classification via LS-SVR. Communications of the Korean Statistical Society, 15, 441-450. https://doi.org/10.5351/CKSS.2008.15.3.441
  12. Shim, J., Park, H. and Hwang, C. (2009). A kernel machine for estimation of mean and volatility functions. Journal of Korean Data & Information Science Society, 20, 905-912.
  13. Shim, J. and Seok, K. H. (2008). Kernel poisson regression for longitudinal data. Journal of Korean Data & Information Science Society, 19, 1353-1360.
  14. Suykens, J. A. K. and Vanderwalle, J. (1995). Least square support vector machine classifier. Neural Processing Letters, 9, 293-300.
  15. Suykens, J. A. K. and Vandewalle, J. (1999). Multiclass least squares support vector machines. Proceeding of the International Joint Conference on Neural Networks, 900-903.
  16. Suykens, J. A. K. (2001). Nonlinear modelling and support vector machines. Proceeding of the Instrumentation and Measurement Technology Conference, 287-294.
  17. Vapnik, V. N. (1995). The nature of statistical learning theory, Springer, New York.
  18. Vapnik, V. N. (1998). Statistical learning theory, Springer, New York.
  19. Weston, J. and Watkins, C. (1998). Multi-class SVM, Technical Report 98-04, Royal Holloway University of London.
  20. Williams, C.K.I. and Seeger, M. (2001). Using the Nystrom method to speed up kernel machines. Advances in Neural Information Processing Systems, 13, 682-699.