Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Semi-supervised regression based on support vector machine

  • Received : 2014.02.13
  • Accepted : 2014.03.06
  • Published : 2014.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In many practical machine learning and data mining applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore semi-supervised learning algorithms have attracted much attentions. However, previous research mainly focuses on classication problems. In this paper, a semi-supervised regression method based on support vector regression (SVR) formulation that is proposed. The estimator is easily obtained via the dual formulation of the optimization problem. The experimental results with simulated and real data suggest superior performance of the our proposed method compared with standard SVR.

Keywords

References

  1. Belkin, M., Niyogi, P. and Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from laveled and unlabeled examples. Journal of Machine Learning Research, 1, 1-48.
  2. Chapelle, O., Scholkopf, B. and Zien, A. (2006). Semi-supervised learning, MIT Press, Cambridge, MA.
  3. Cortes, C. and Mohri, M. (2007). On transductive regression. In Advances in Neural Information Processing System, 19, 305-312.
  4. Cristianini, N. and Shawe-Taylor, J. (2000). An introduction to support vector machines, Cambridge University Press, United Kingdom.
  5. Kuhn, H. and Tucker, A. (1951) Nonlinear programming. In Proceedings of 2nd Berkeley Symposium, University of California Press, Berkeley, 481-492.
  6. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 2, Morgan Kaufmann, San Mateo, CA, 1137-1143.
  7. Lafferty, J. and Wasserman, L. (2008). Statistical analysis of semi-supervised regression. In Advances in Neural Information Processing Systems, 20, 801-808.
  8. Mercer, J. (1909). Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London A, 209, 415-446. https://doi.org/10.1098/rsta.1909.0016
  9. Niyogi, P. (2008). Manifold regularization and semi-supervised learning: Some theoretical analyses, Technical Report TR-2008-01, Computer Science Department, University of Chicago, Chicago, IL.
  10. Rosipal, R. and Trejo, L. J. (2001). Kernel partial least squares regression in reproducing kernel Hilbert space. Journal of Machine Learning Research, 2, 97-123
  11. Seok, K. (2010). Semi-supervised classification with LS-SVM formulation. Journal of Korean Data & Information Science Society, 21, 461-470.
  12. Seok, K. (2012). Study on semi-supervised local constant regression estimation. Journal of the Korean Data & Information Science Society, 23, 579-585. https://doi.org/10.7465/jkdi.2012.23.3.579
  13. Seok, K. (2013). A study on semi-supervised kernel ridge regression estimation. Journal of the Korean Data & Information Science Society, 24, 341-353. https://doi.org/10.7465/jkdi.2013.24.2.341
  14. Shim, J. and Hwang, C. (2009). Support vector censored quantile regression under random censoring. Computational Statistics and Data Analysis, 53, 912-919. https://doi.org/10.1016/j.csda.2008.10.037
  15. Singh, A., Nowak, R. and Zhu, X. (2008). Unlabeled data: Now it helps, now it doesn't. In Advances in Neural Information Processing Systems, 21, 1513-1520.
  16. Smola, A. and Scholkopf, B. (1998). On a kernel-based method for pattern recognition, regression, approximation and operator inversion. Algorithmica, 22, 211-231. https://doi.org/10.1007/PL00013831
  17. Vapnik, V. N. (1995). The nature of statistical learning theory, Springer, New York.
  18. Vapnik, V. N. (1998). Statistical learning theory, Wiley, New York.
  19. Wang, M., Hua, X., Song, Y., Dai, L. and Zhang, H. (2006). Semi-supervised kernel regression. In Proceeding of the Sixth IEEE International Conference on Data Mining, 1130-1135.
  20. Xu, S., An. X., Qiao, X., Zhu, L. and Li, L. (2011). Semisupervised least squares support vector regression machines. Journal of Information & Computational Science, 8, 885-892.
  21. Zhou, Z. and Li, M. (2007). Semi-supervised regression with co-training style algorithm. IEEE Transactions on Knowledge and Data Engineering, 19, 1479-1493. https://doi.org/10.1109/TKDE.2007.190644
  22. Zhu, D. (2005). Semi-supervised learning literature survey, Technical Report, Computer Sciences Department, University of Wisconsin, Madison, WI.
  23. Zhu, X. and Goldberg, A. (2009). Introduction to semi-supervised learning, Morgan & Claypool, London.

Cited by

  1. Semisupervised support vector quantile regression vol.26, pp.2, 2015, https://doi.org/10.7465/jkdi.2015.26.2.517