Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Support vector quantile regression ensemble with bagging

  • Received : 2014.04.27
  • Accepted : 2014.05.18
  • Published : 2014.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Support vector quantile regression (SVQR) is capable of providing more complete description of the linear and nonlinear relationships among random variables. To improve the estimation performance of SVQR we propose to use SVQR ensemble with bagging (bootstrap aggregating), in which SVQRs are trained independently using the training data sets sampled randomly via a bootstrap method. Then, they are aggregated to obtain the estimator of the quantile regression function using the penalized objective function composed of check functions. Experimental results are then presented, which illustrate the performance of SVQR ensemble with bagging.

Keywords

References

  1. Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123-140.
  2. Efron, B. and Tibshirani, R. (1993). An introduction to the bootstrap, Chapman & Hall/CRC, Boca Raton.
  3. Hwang, C. (2010). M-quantile regression using kernel machine technique. Journal of the Korean Data & Information Science Society, 21, 973-981.
  4. Hwang, C. and Shim, J. (2005). A simple quantile regression via support vector machine. Lecture Notes in Computer Science, 3610, 512-520. https://doi.org/10.1007/11539087_66
  5. Hwang, C. and Shim, J. (2012). Smoothing Kaplan-Meier estimate using monotone suppot vector regression. Journal of the Korean Data & Information Science Society, 23, 1045-1054. https://doi.org/10.7465/jkdi.2012.23.6.1045
  6. Kim, H., Pang, S., Je, H., Kim D. and Bang, S. Y. (2003). Constructing support vector machine ensemble. Pattern Recognition, 36, 2357-2767.
  7. Koenker, R. (2005). Quantile regression, Cambridge University Press, Cambridge.
  8. Koenker, R. and Bassett, G. (1978). Regression quantile. Econometrica, 46, 33- 50. https://doi.org/10.2307/1913643
  9. Koenker, R. and Hallock, K. F. (2001). Quantile regression. Journal of Economic Perspectives, 40, 122-142.
  10. Kuhn, H. W. and Tucker, A. W. (1951). Nonlinear programming. In Proceedings of 2nd Berkeley Symposium, University of California Press, Berkeley, 481- 492.
  11. Lee, J. and Shim, J. (2010). Restricted support vector regression without crossing. Journal of the Korean Data & Information Science Society, 21, 319-325.
  12. Li, Y., Liu, Y. and Zhu, J. (2007). Quantile regression in reproducing kernel Hilbert spaces. Journal of the American Statistical Association, 102, 255-268. https://doi.org/10.1198/016214506000000979
  13. Mercer, J. (1909). Functions of positive and negative type and their connection with theory of integral equations. Philosophical Transactions of Royal Society A, 415-446.
  14. Saunders, C., Gammerman, A. and Vovk, V. (1998). Ridge regression learning algorithm in dual variables. In Proceedings of the 15th International Conference on Machine Learning, 515-521.
  15. Schapire, R., Freund, Y., Bartlett, P. and Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26, 824-832.
  16. Shim, J. and Hwang, C. (2012). M-quantile kernel regression for small area estimation. Journal of the Korean Data & Information Science Society, 23, 749-756. https://doi.org/10.7465/jkdi.2012.23.4.749
  17. Shim, J. and Hwang, C. (2013). Expected shortfall estimation using kernel machines Journal of the Korean Data & Information Science Society, 24, 625-636. https://doi.org/10.7465/jkdi.2013.24.3.625
  18. Shim, J., Seok, K. H. and Hwang, C. (2009). Non-crossing quantile regression via doubly penalized kernel machine. Computational Statistics, 24, 83-94. https://doi.org/10.1007/s00180-008-0123-y
  19. Takeuchi, I., Le, Q. V., Sears, T. D. and Smola, A. J. (2006). Nonparametric quantile estimation. Journal of Machine Learning Research, 7, 1231-1264.
  20. Vapnik, V. N. (1995). The nature of statistical learning theory, Springer, New York.
  21. Yu, K., Lu, Z. and Stander, J. (2003). Quantile regression: Applications and current research area. The Statistician, 52, 331-350.
  22. Yuan, M. (2006). GACV for quantile smoothing splines. Computational Statistics & Data Analysis, 50, 813-829. https://doi.org/10.1016/j.csda.2004.10.008

Cited by

  1. Tree size determination for classification ensemble vol.27, pp.1, 2016, https://doi.org/10.7465/jkdi.2016.27.1.255
  2. Multioutput LS-SVR based residual MCUSUM control chart for autocorrelated process vol.27, pp.2, 2016, https://doi.org/10.7465/jkdi.2016.27.2.523