Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

A Note on Linear SVM in Gaussian Classes

  • Jeon, Yongho (Department of Applied Statistics, Yonsei University)
  • Received : 2013.04.07
  • Accepted : 2013.04.27
  • Published : 2013.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

The linear support vector machine(SVM) is motivated by the maximal margin separating hyperplane and is a popular tool for binary classification tasks. Many studies exist on the consistency properties of SVM; however, it is unknown whether the linear SVM is consistent for estimating the optimal classification boundary even in the simple case of two Gaussian classes with a common covariance, where the optimal classification boundary is linear. In this paper we show that the linear SVM can be inconsistent in the univariate Gaussian classification problem with a common variance, even when the best tuning parameter is used.

Keywords

References

  1. Bartlett, P., Jordan, M. and McAuliffe, J. (2006). Convexity, classification, and risk bounds, Journal of the American Statistical Association, 101, 138-156. https://doi.org/10.1198/016214505000000907
  2. Blanchard, G., Bousquet, O. and Massart, P. (2008). Statistical performance of support vector machines, The Annals of Statistics, 36, 489-531. https://doi.org/10.1214/009053607000000839
  3. Boser, B., Guyon, I. and Vapnik, V. (1992). A training algorithm for optimal margin classifiers, In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 144-152, ACM.
  4. Cortes, C. and Vapnik, V. (1995). Support-vector networks, Machine Learning, 20, 273-297.
  5. Cristianini, N. and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University press.
  6. Koo, J., Lee, Y., Kim, Y. and Park, C. (2008). A bahadur representation of the linear support vector machine, The Journal of Machine Learning Research, 9, 1343-1368.
  7. Li, K.-C. and Duan, N. (1989). Regression analysis under link violation, The Annals of Statistics, 17, 1009-1052. https://doi.org/10.1214/aos/1176347254
  8. Lin, Y. (2000). Some asymptotic properties of the support vector machine, Technical Report 1029, Department of Statistics, University of Wisconsin-Madison.
  9. Lin, Y. (2004). A note on margin-based loss functions in classification, Statistics & Probability Letters, 68, 73-82. https://doi.org/10.1016/j.spl.2004.03.002
  10. Scholkopf, B. and Smola, A. (2001). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT press.
  11. Steinwart, I. (2005). Consistency of support vector machines and other regularized kernel classifiers, IEEE Transactions on Information Theory, 51, 128-142. https://doi.org/10.1109/TIT.2004.839514
  12. Steinwart, I. and Scovel, C. (2007). Fast rates for support vector machines using Gaussian kernels, The Annals of Statistics, 35, 575-607. https://doi.org/10.1214/009053606000001226
  13. Vapnik, V. (1999). The Nature of Statistical Learning Theory, Springer.
  14. Xu, H., Caramanis, C. and Mannor, S. (2009). Robustness and regularization of support vector machines, The Journal of Machine Learning Research, 10, 1485-1510.
  15. Zhang, T. (2004). Statistical behavior and consistency of classification methods based on convex risk minimization, Annals of Statistics, 32, 56-85.