JOURNAL BROWSE
Search
Advanced SearchSearch Tips
On the Use of Adaptive Weights for the F-Norm Support Vector Machine
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
On the Use of Adaptive Weights for the F-Norm Support Vector Machine
Bang, Sung-Wan; Jhun, Myoung-Shic;
  PDF(new window)
 Abstract
When the input features are generated by factors in a classification problem, it is more meaningful to identify important factors, rather than individual features. The -norm support vector machine(SVM) has been developed to perform automatic factor selection in classification. However, the -norm SVM may suffer from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each factor without assessing its relative importance. To overcome such a limitation, we propose the adaptive -norm (-norm) SVM, which penalizes the empirical hinge loss by the sum of the adaptively weighted factor-wise -norm penalty. The -norm SVM computes the weights by the 2-norm SVM estimator and can be formulated as a linear programming(LP) problem which is similar to the one of the -norm SVM. The simulation studies show that the proposed -norm SVM improves upon the -norm SVM in terms of classification accuracy and factor selection performance.
 Keywords
Adaptive weight;-norm penalty;factor selection;feature selection;support vector machine;
 Language
English
 Cited by
1.
Hierarchically penalized support vector machine for the classication of imbalanced data with grouped variables, Korean Journal of Applied Statistics, 2016, 29, 5, 961  crossref(new windwow)
 References
1.
Bang, S. and Jhun, M. (2012a). Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization, Computational Statistics and Data Analysis, 56, 813-826 . crossref(new window)

2.
Bang, S. and Jhun, M. (2012b). Adaptive sup-norm regularized simultaneous multiple quantiles regression, Statistics, accepted for publication.

3.
Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. crossref(new window)

4.
Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning, Springer-Verlag, New York.

5.
Hoerl, A. and Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems, Technometrics, 12, 55-67. crossref(new window)

6.
Kaufman, L. and Rousseeuw, P. J. (1990). Finding Groups in Data: An Introduction to Cluster Analysis, Wiley, New York.

7.
Leng, C., Lin, Y. and Wahba, G. (2006). A note on lasso and related procedures in model selection, Statistica Sinica, 16, 1273-1284.

8.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288.

9.
Vapnik, V. (1995). The Nature of Statistical Learning Theory, Springer-Verlag, New York.

10.
Wang, H. and Leng, C. (2008). A note on adaptive group lasso, Computational Statistics and Data Analysis, 52, 5277-5286. crossref(new window)

11.
Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society, Series B, 68, 49-67. crossref(new window)

12.
Yuan, M. and Lin, Y. (2007). On the nonnegative garrote estimator, Journal of the Royal Statistical Society, Series B, 69, 143-161. crossref(new window)

13.
Zhang, H., Liu, Y., Wu, Y. and Zhu, J. (2008). Variable selection for multicategory svm via sup-norm regularization, Electronic Journal of Statistics, 2, 149-167. crossref(new window)

14.
Zhao, P., Rocha, G. and Yu, B. (2009). The composite absolute penalties family for grouped and hierarchical variable selection, The Annals of Statistics, 37, 3468-3497. crossref(new window)

15.
Zhu, J., Rosset, S., Hastie, T. and Tibshirani, R. (2003). 1-norm support vector machine, Neural Information Proceeding Systems, 16.

16.
Zou, H. (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429. crossref(new window)

17.
Zou, H. (2007). An improved 1-norm SVM for simultaneous classification and variable selection, In Proceedings of the 11th International Conference on Arti cial Intelligence and Statistics.

18.
Zou, H. and Yuan, M. (2008). The $F_{\infty}$-norm support vector machine, Statisitca Sinica, 18, 379-398.