Genetic Outlier Detection for a Robust Support Vector Machine



Lee, Heesung;Kim, Euntai

  • 투고 : 2015.04.08
  • 심사 : 2015.06.25
  • 발행 : 2015.09.30


Support vector machine (SVM) has a strong theoretical foundation and also achieved excellent empirical success. It has been widely used in a variety of pattern recognition applications. Unfortunately, SVM also has the drawback that it is sensitive to outliers and its performance is degraded by their presence. In this paper, a new outlier detection method based on genetic algorithm (GA) is proposed for a robust SVM. The proposed method parallels the GA-based feature selection method and removes the outliers that would be considered as support vectors by the previous soft margin SVM. The proposed algorithm is applied to various data sets in the UCI repository to demonstrate its performance.


SVM;Robust SVM;Genetic algorithm;Support vectors;Outlier


  1. Q. Song, W. Hu, and W. Xie, “Robust support vector machine with bullet hole image classification,” IEEE Trans. Systems, Man, and Cybernetics-Part C: Applications and Reviews, vol. 32, no. 4, pp. 440–448, 2002.
  2. N. Krause and Y. Singer, “Leveraging the margin more carefully,” in Proc. the 21st International Conference on Machine Learning, vol. 69, 2004.
  3. P. Bartlett and S. Mendelson, “Rademacher and Gaussian complexities: risk bounds and structural results,” Journal of Machine Learning Research, vol. 3, pp. 463-482, 2002.
  4. L. Davis, Handbook of Genetic Algorithms. Van Nostrand Reinhold, 1991.
  5. H. Lee, E. Kim, and M. Park, “A genetic feature weighting scheme for pattern recognition,” Integrated Computer-Aided Engineering, vol. 14, pp. 161-171, 2007.
  6. L. Kuncheva and L. Jain, “Nearest neighbor classifier: simultaneousediting and feature selection,” Pattern Recognition Letters, vol. 20, pp. 1149-1156, 1999.
  7. I. Oh, J. Lee, and B. Moon, “Hybrid genetic algorithms for feature selection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 11, pp. 1424-1437, 2004.
  8. H. Juo and H. Chang, “A new symbiotic evolution-based fuzzy-neural approach to fault diagnosis of marine propulsion systems,” Artificial Intelligence, vol. 17, pp. 919-930, 2004.
  9. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. Springer, 1996.
  10. P. M. Murphy and D. W. Aha, “UCI Repository for Machine Learning Databases,” Technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, Calif., 1994.
  11. S. Jun, “An outlier data analysis using support vector regression,” Journal of The Korean Institute of Intelligent Systems, vol. 18, no. 6, pp. 876-880, 2008.
  12. V. Hoang, M. Le, and K. Jo, “Hybrid cascade boosting machine using variant scale blocks based HOG features for pedestrian detection,” Neurocomputing, vol. 135, pp. 357-366, 2014.
  13. S. Seo, H. Yang, K. Sim, “Behavior learning and evolution of swarm robot system using support vector machine,” Journal of The Korean Institute of Intelligent Systems, vol. 18, no. 5, pp. 712-717, 2008.
  14. H. Shin, H. Jung, K. Cho, and J. Lee “A prediction method of learning outcomes based on regression model for effective peer review learning,” Journal of The Korean Institute of Intelligent Systems, vol. 22, no. 5, pp. 624-630, 2012.
  15. S. Kumar, Neural Networks: A Classroom Approach. McGraw-Hill, 2005.
  16. L. Wang, H. Jia, and J. Li, “Training robust support vector machine with smooth ramp loss in primal space,” Neurocomputing, vol. 71, pp. 3020-3025, 2008.
  17. H. Lee, S. Hong, B. Lee and E. Kim “Design of robust support vector machine using genetic algorithm,” Journal of The Korean Institute of Intelligent Systems, vol. 20, no. 3, pp. 375-379, 2010.
  18. L. Xu, K. Crammer, and D. Schuurmans, “Robust support vector machine training via convex outlier ablation,” in Proc. the 21st National Conference on Artificial Intelligence, pp. 536-542, 2006.
  19. J. A. K. Suykens, and J. Vandewalle, “Least squares support vector machine classifiers,” Neural Processing Letters, vol. 9, no. 3, pp. 293-300, 1999.
  20. C. Cortes, and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273-297, Sep. 1995.
  21. V. N. Vapnik, Statistical Learning Theory. Wiley, 1998.