The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Divide and conquer kernel quantile regression for massive dataset

대용량 자료의 분석을 위한 분할정복 커널 분위수 회귀모형

  • Bang, Sungwan (Department of Mathematics, Korea Military Academy) ;
  • Kim, Jaeoh (Center for Army Analysis and Simulation, HQs ROKA)
  • Received : 2020.06.15
  • Accepted : 2020.07.27
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

By estimating conditional quantile functions of the response, quantile regression (QR) can provide comprehensive information of the relationship between the response and the predictors. In addition, kernel quantile regression (KQR) estimates a nonlinear conditional quantile function in reproducing kernel Hilbert spaces generated by a positive definite kernel function. However, it is infeasible to use the KQR in analysing a massive data due to the limitations of computer primary memory. We propose a divide and conquer based KQR (DC-KQR) method to overcome such a limitation. The proposed DC-KQR divides the entire data into a few subsets, then applies the KQR onto each subsets and derives a final estimator by aggregating all results from subsets. Simulation studies are presented to demonstrate the satisfactory performance of the proposed method.

분위수 회귀모형은 반응변수의 조건부 분위수 함수를 추정함으로써 반응변수와 예측변수의 관계에 대한 포괄적인 정보를 제공한다. 특히 커널 분위수 회귀모형은 비선형 관계식을 고려하기 위하여 양정치 커널함수(kernel function)에 의해 만들어지는 재생 커널 힐버트 공간(reproducing kernel Hilbert space)에서 비선형 조건부 분위수 함수를 추정한다. 그러나 KQR은 이차계획법으로 공식화되어 많은 계산비용을 필요로 하므로 컴퓨터 메모리 능력의 제한으로 대용량 자료의 분석은 불가능하다. 이러한 문제점을 해결하기 위하여 본 논문에서는 분할정복(divide and conquer) 알고리즘을 활용한 KQR 추정법(DC-KQR)을 제안한다. DC-KQR은 먼저 전체 훈련자료를 몇 개의 부분집합으로 무작위로 분할(divide)한 후, 각각의 부분집합에 대하여 KQR 분위수 함수를 추정하고 이들의 산술 평균을 이용하여 최종적인 추정량으로 통합(conquer)하는 기법이다. 본 논문에서는 모의실험과 실제자료 분석을 통해 제안한 DC-KQR의 효율적인 성능과 활용 가능성을 확인하였다.

Keywords

References

  1. Bang, S., Eo, S-H., Cho, Y., Jhun, M., and Cho, H. (2016). Non-crossing weighted kernel quantile regression with right censored data, Lifetime Data Analysis, 22, 100-121. https://doi.org/10.1007/s10985-014-9314-8
  2. Bang, S. and Shin, S. (2016). A comparison study of multiple linear quantile regression using non-crossing constraints, The Korean Journal of Applied Statistics, 29, 773-786. https://doi.org/10.5351/KJAS.2016.29.5.773
  3. Bertin-Mahieux, T., Ellis, D., Whitman, B., and Lamere, P. (2011). The million song dataset, In Proceedings of the 12th International Conference on Music Information Retrieval(IS-MIR).
  4. Caputo, B., Sim, K., Furesjo, F., and Smola, A. (2002). Appearance-based object recognition using SVMs: Which kernel should I use?. In Proceedings of INPS workshop on Statistical methods for computational Experiments in Visual Processing and Computer Vision, 149-158.
  5. Chen, X., Liu, W., and Zhang, Y. (2018). Quantile regression under memory constraint, arXiv preprint arXiv:1810.08264.
  6. Chen, L. and Zhou, Y. (2020). Quantile regression in big data: A divide and conquer based strategy, Computational Statistics and Data Analysis, 144, 1-17.
  7. Chen, X., and Xie, M. G. (2014). A split-and-conquer approach for analysis of extraordinarily large data, Statistica Sinica, 24, 1655-1684.
  8. Cole, T. and Green, P. (1992). Smoothing Reference Centile Curves: The LMS Method and Penalized Likelihood, Statistics in Medicine, 11, 1305-1319. https://doi.org/10.1002/sim.4780111005
  9. Dhillon, I., Guan, Y., and Kulis, B. (2004). Kernel k-means, spectral clustering and normalized cuts, KDD 2004, 551-556.
  10. Fan, T., Lin, D., and Cheng, K. (2007). Regression analysis for massive datasets, Data and Knowledge Engineering, 61, 554-562. https://doi.org/10.1016/j.datak.2006.06.017
  11. Heagerty, P. and Pepe, M. (1999). Semiparametric Estimation of Regression Quantiles with Application to Standardizing Weight for Height and Age in U.S. Children, The Journal of the Royal Statistical Society, Series C (Applied Statistics), 48, 533-551. https://doi.org/10.1111/1467-9876.00170
  12. Jiang, R., Hu, X., Yu, K., and Qian, W. (2018). Composite quantile regression for massive datasets, Statistics, 52, 980-1004. https://doi.org/10.1080/02331888.2018.1500579
  13. Kang, J. and Jhun, M. (2020). Divide-and-conquer random sketched kernel ridge regression for large-scale data, Journal of the Korean Data & Information Science Society, 31, 15-23. https://doi.org/10.7465/jkdi.2020.31.1.15
  14. Karatzoglou, A., Smola, A., Hornik, K., and Zeileis, A. (2004). kernlab-An S4 package for kernel methods in R, Journal of Statistical Software, 11, 1-20.
  15. Kimeldorf, G. and Wahba, G. (1971). Some results on Tchebycheffian spline functions, Journal of Mathematical Analysis and Applications, 33, 82-95. https://doi.org/10.1016/0022-247X(71)90184-3
  16. Koenker, R. and Bassett, G. (1978). Regression quantiles, Econometrica, 4, 33-50. https://doi.org/10.2307/1913643
  17. Koenker, R. and Geling, R. (2001). Reappraising Medfly Longevity: A Quantile Regression Survival Analysis, Journal of the American Statistical Association, 96, 458-468. https://doi.org/10.1198/016214501753168172
  18. Koenker, R. and Hallock, K. (2001). Quantile Regression, Journal of Economic Perspectives, 15, 143-156. https://doi.org/10.1257/jep.15.4.143
  19. Li, Y., Liu, Y., and Zhu, J. (2007). Quantile regression in reproducing kernel Hilbert spaces, Journal of the American Statistical Association, 102, 255-268. https://doi.org/10.1198/016214506000000979
  20. Li, Y. and Zhu, J. (2008). L1-norm quantile regression, Journal of Computational and Graphical Statistics, 17, 1-23. https://doi.org/10.1198/106186008X287328
  21. Lin, N. and Xi, R. (2011). Aggregated estimating equation estimation, Statistics and Its Interface, 4, 73-83. https://doi.org/10.4310/SII.2011.v4.n1.a8
  22. Powell, D. and Wagner, J. (2014). The exporter productivity premium along the productivity distribution: evidence from quantile regression with nonadditive firm fixed effects, Review of World Economics, 150, 763-785. https://doi.org/10.1007/s10290-014-0192-7
  23. Vapnik, V. N. (1998). Statistical Learning Theory, Wiley, New York.
  24. Wang, H. and He, X. (2007). Detecting differential expressions in genechip microarray studies: A quantile approach, Journal of the American Statistical Association, 102, 104-112. https://doi.org/10.1198/016214506000001220
  25. Wu, Y. and Liu, Y. (2009). Stepwise multiple quantile regression estimation using non-crossing constraints, Statistics and Its Interface, 2, 299-310. https://doi.org/10.4310/SII.2009.v2.n3.a4
  26. Xu, Q., Cai, C., Jiang, C., Sun, F., and Huang, X. (2020). Block average quantile regression for massive dataset, Statistical Papers, 61, 141-165. https://doi.org/10.1007/s00362-017-0932-6
  27. Yang, H. and Liu, H. (2016). Penalized weighted composite quantile estimators with missing covariates, Statistical Papers, 57, 69-88. https://doi.org/10.1007/s00362-014-0642-2
  28. Zhang, Y., Duchi, J., andWainwright, M. (2015). Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates, Journal of Machine Learning Research, 16, 3299-3340.