Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

A study on semi-supervised kernel ridge regression estimation

준지도 커널능형회귀모형에 관한 연구

  • 석경하 (인제대학교 데이터정보학과)
  • Received : 2013.02.21
  • Accepted : 2013.03.18
  • Published : 2013.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In many practical machine learning and data mining applications, unlabeled data are inexpensive and easy to obtain. Semi-supervised learning try to use such data to improve prediction performance. In this paper, a semi-supervised regression method, semi-supervised kernel ridge regression estimation, is proposed on the basis of kernel ridge regression model. The proposed method does not require a pilot estimation of the label of the unlabeled data. This means that the proposed method has good advantages including less number of parameters, easy computing and good generalization ability. Experiments show that the proposed method can effectively utilize unlabeled data to improve regression estimation.

데이터마이닝과 기계학습의 응용분야에서는 라벨 없는 자료를 이용하는 연구가 많이 진행되고 있다. 이러한 연구는 분류문제에 집중되었다가 최근에 회귀분석문제로 관심이 모아지고 있다. 본 연구에서는 커널능형회귀모형 형태의 준지도 회귀분석 방법을 제시한다. 제안된 방법은 기존의 전환적 방법과는 달리 라벨 없는 자료의 라벨을 추정하는 과정을 필요로 하지 않기 때문에 선택해야 할 모수의 수도 적고, 계산과정도 단순할 뿐 아니라 일반화에 강점이 있다. 모의실험과 실제 자료 분석을 통해 제안된 방법이 라벨 없는 자료를 잘 활용하여 라벨 있는 자료만 이용하는 방법보다 더 우수한 추정을 하는 것을 볼 수 있었다.

Keywords

References

  1. Belkin, M., Niyogi, P. and Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from laveled and unlabeled examples. Journal of Machine Learning Research, 1, 1-48.
  2. Chapelle, O., Scholkopf, B. and Zien, A. (2006). Semi-supervised learning, MIT Press, Cambridge, MA.
  3. Cortes, C. and Mohri, M. (2007). On transductive regression. In Advances in Neural Information Processing System, 19, 305-312.
  4. Exterkate, P. (2012). Model selection in kernel ridge regression, CREATES Research Papers from School of Economics and Management, University of Aarhus, Aarhus, Denmark.
  5. Hofmann, T., Scholkopf, B. and Smola, A. J. (2008) Kernel methods in machine learning. Annals of Statistics, 36, 1171-1220. https://doi.org/10.1214/009053607000000677
  6. Kimeldorf, G. S. and Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications, 33, 82-95. https://doi.org/10.1016/0022-247X(71)90184-3
  7. Lafferty, J. and Wasserman, L. (2008). Statistical analysis of semi-supervised regression. In Advances in Neural Information Processing Systems, 20, 801-808.
  8. Lin, H. and Lin, C. (2003). A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods, Technical Report, Department of Computer Science, National Taiwan University, Taipei, Taiwan.
  9. Mercer, J. (1909). Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London A, 209, 415-446. https://doi.org/10.1098/rsta.1909.0016
  10. Nigam, K. and Ghani, R. (2000). Analyzing the effectiveness and applicability of co-training. Ninth International Conference on Information and Knowledge Management, 86-93.
  11. Niyogi, P. (2008). Manifold regularization and semi-supervised learning: Some theoretical analyses, Technical Report TR-2008-01, Computer Science Department, University of Chicago, Chicago, IL.
  12. Rosenberg, C., Hebert, M. and Schneiderman, H. (2005). Semi-supervised self-training of object detection models. Seventh IEEE Workshop on Applications of Computer Vision, 1, 29-36.
  13. Rosipal, R. and Trejo, L. (2001) Kernel partial least squares regression in reproducing kernel Hilbert space. Journal of Machine Learning Research, 2, 1667-1689.
  14. Seok, K. (2012). Study on semi-supervised local constant regression estimation. Journal of the Korean Data & Information Science Society, 23, 579-585. https://doi.org/10.7465/jkdi.2012.23.3.579
  15. Singh, A., Nowak, R. and Zhu, X. (2008). Unlabeled data: Now it helps, now it doesn't. In Advances in Neural Information Processing Systems, 21, 1513-1520.
  16. Suykens, J.A.K., Gastel, T. V., Bravanter, J. D., Moore, B. D. and Vandewalle, J. (2002). Least squares support vector machines, World Scientific, London.
  17. Szummer, M. and Jaakkola, T. (2002). Information regularization with partially labeled data. In Advances in Neural Information Processing Systems, 15, 1025-1032.
  18. Ungar, L. H. (1995). UPenn ChemData repository [Machine-readable data repository], Philadelphia, PA. Available electronically via ftp://ftp.cis.upenn.edu/pub/ungar/chemdata.
  19. Vapnik, V. (1998). Statistical learning theory, Wiley, New York.
  20. Wang, M., Hua, X., Song, Y., Dai, L. and Zhang, H. (2006). Semi-supervised kernel regression. In Proceeding of the Sixth IEEE International Conference on Data Mining, 1130-1135.
  21. Xu, S., An. X., Qiao, X., Zhu, L. and Li, L. (2011) Semisupervised least squares support vector regression machines. Journal of Information & Computational Science, 8, 885-892.
  22. Xu, Z., King, I. and Lyu, M. R. (2010). More than semi-supervised learning, LAP LAMBERT Academic Publishing, London.
  23. Zhou, X., Ghahramani, Z. and Lafferty, J. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. In Proceedings of the 20th International Conference on Machine Learning, 912-919.
  24. Zhou, Y. and Goldman, S. (2004). Democratic co-learing. In Proceedings of the16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI2004), 594-602.
  25. Zhou, Z. and Li, M. (2007). Semi-supervised regression with co-training style algorithm. IEEE Transactions on Knowledge and Data Engineering, 19, 1479-1493. https://doi.org/10.1109/TKDE.2007.190644
  26. Zhu, D. (2005). Semi-supervised learning literature survey, Technical Report, Computer Sciences Department, University of Wisconsin, Madison, WI.
  27. Zhu, X. and Goldberg, A. (2009). Introduction to semi-supervised learning, Morgan & Claypool, London.

Cited by

  1. Smoothing parameter selection in semi-supervised learning vol.27, pp.4, 2016, https://doi.org/10.7465/jkdi.2016.27.4.993
  2. Semisupervised support vector quantile regression vol.26, pp.2, 2015, https://doi.org/10.7465/jkdi.2015.26.2.517
  3. A transductive least squares support vector machine with the difference convex algorithm vol.25, pp.2, 2014, https://doi.org/10.7465/jkdi.2014.25.2.455
  4. Semi-supervised regression based on support vector machine vol.25, pp.2, 2014, https://doi.org/10.7465/jkdi.2014.25.2.447