The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Hybrid Genetic Algorithm for Classifier Ensemble Selection

분류기 앙상블 선택을 위한 혼합 유전 알고리즘

  • 김영원 (한국전자통신연구원 우정기술연구센터) ;
  • 오일석 (전북대학교 전자정보공학부)
  • Published : 2007.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a hybrid genetic algorithm(HGA) for the classifier ensemble selection. HGA is added a local search operation for increasing the fine-turning of local area. This paper apply hybrid and simple genetic algorithms(SGA) to the classifier ensemble selection problem in order to show the superiority of HGA. And this paper propose two methods(SSO: Sequential Search Operations, CSO: Combinational Search Operations) of local search operation of hybrid genetic algorithm. Experimental results show that the HGA has better searching capability than SGA. The experiments show that the CSO considering the correlation among classifiers is better than the SSO.

이 논문은 최적의 분류기 앙상블 선택을 위한 혼합 유전 알고리즘을 제안한다. 혼합 유전 알고리즘은 단순 유전알고리즘의 미세 조정력을 보완하기 위해 지역 탐색 연산을 추가한 것이다. 혼합 유전 알고리즘의 우수성을 입증하기 위해 단순 유전 알고리즘과 혼합 유전 알고리즘 각각을 비교 실험하였다. 또한 혼합 유전 알고리즘의 지역 탐색 연산으로 두 가지 방법(SSO: 순차 탐색 연산, CSO: 조합 탐색 연산)을 제안한다. 비교 실험 결과는 혼합 유전 알고리즘이 단순 유전 알고리즘에 비해 해를 탐색하는 능력이 우수하였다. 또한 분류기들의 상관관계를 고려한 CSO 방법이 SSO 방법보다 더 우수하였다.

Keywords

References

  1. P.M. Granitto, P.F. Verdes, and H.A. Ceccatto, 'Neural network ensembles: evaluation of aggregation algorithms,' Artificial Intelligence, Vol.163, No.2, pp.139-162, 2005 https://doi.org/10.1016/j.artint.2004.09.006
  2. Zhi-Hua Zhou, Jianxin Wu, and Wei Tang, 'Ensembling neural networks: many could be better than all,' Artificial Intelligence, Vol.137, pp.239-263, 2002 https://doi.org/10.1016/S0004-3702(02)00190-X
  3. Chanho Park and Sung-Bae Cho, 'EVolutionary ensemble classifier for lymphoma and colon cancer classification,' Proc. of IEEE International Conf. on EVolutionary Computation, Vol.4, pp.2378-2385, 2003 https://doi.org/10.1109/CEC.2003.1299385
  4. N. Ueda, 'Optimal linear combination of neural networks for improving classification performance,' IEEE Tr. Pattern Analysis and Machine Intelligence, Vol.22, No.2, pp.207-215, 2000 https://doi.org/10.1109/34.825759
  5. L. Breiman, 'Bagging predictors,' Machine Learning, Vol.24, No.2, pp.123-140, 1996 https://doi.org/10.1007/BF00058655
  6. Robert Bryll, Ricardo Gutierrez-Osuna, and Francis Quek, 'Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets,' Pattern Recognition, Vol.36, No.6, pp.1291-1302, 2003 https://doi.org/10.1016/S0031-3203(02)00121-8
  7. J. R. Quinlan, 'Bagging, boosting, and C4.5,' Proc. AAAI-96, pp.725-730, 1996
  8. Tin Kam Ho, 'Multiple classifier combination: lessons and next steps,' in Hybrid Methods in Pattern Recognition, (Ed. by H. Bubke & A. Kandel), World Scientific, 2002
  9. Tin Kam Ho, 'The random subspace method for constructing decision forests,' IEEE Tr. Pattern Analysis and Machine Intelligence, Vol.20, No.8, pp.832-844, 1998 https://doi.org/10.1109/34.709601
  10. G. Tremblay, R. Sabourin, and P. Maupin, 'Optimizing nearest neighbour in random subspaces using a multi-objective genetic algorithm,' In Proc. of 17th ICPR, Vol.1, pp.208-211, 2004 https://doi.org/10.1109/ICPR.2004.662
  11. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas, 'On combining classifiers,' IEEE Tr. Pattern Analysis and Machine Intelligence, Vol.20, No.3, pp.226-239, 1998 https://doi.org/10.1109/34.667881
  12. Antanas Verikas, Arunas Lipnickas, Kerstin Malmqvist, Marija Bacauskiene, and Adas Gelzinis, 'Soft combination of neural classifiers: a comparative study,' Pattern Recognition Letters, Vol.20, No.4, pp429-444, 1999 https://doi.org/10.1016/S0167-8655(99)00012-4
  13. S. Wesolkowski and K. Hassanein, 'A comparative study of combination schemes for an ensemble of digit recognition neural networks,' Proc. of IEEE International Conf. on Computational Cybernetics and Simulation. Vol.4, pp.3534-3539, 1997 https://doi.org/10.1109/ICSMC.1997.633202
  14. G. Fumera and F. Roli, 'A theoretical and experimental analysis of linear combiners for multiple classifier systems,' IEEE Tr. Pattern Analysis and Machine Intelligence, Vol.27, No.6, pp.942-956, 2005 https://doi.org/10.1109/TPAMI.2005.109
  15. Hee- Joong Kang and David Doermann, 'Selection of classifiers for the construction of multiple classifier systems,' Proc. of International Conf. on Document Analysis and Recognition, Vol.2, pp.1194-1198, 2005 https://doi.org/10.1109/ICDAR.2005.213
  16. Il-Seok Oh, jin-Seon Lee, and Byung-Ro Moon, 'Hybrid genetic algorithms for feature selection,' IEEE Tr. Pattern Analysis and Machine Intelligence, Vol.26, No.11, pp.1424-1437, 2004 https://doi.org/10.1109/TPAMI.2004.105
  17. P. Jog, J. Suh, and D. Gucht, 'The effect of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem,' Proc. of International Conference on Genetic Algorithms, pp.110-115, 1989
  18. T.N. Bui and B.R. Moon, 'Genetic algorithm and graph partitioning,' IEEE Tr. Computers, Vol.45, No.7, pp.841-855, 1996 https://doi.org/10.1109/12.508322
  19. X. Zheng, B.A. Julstrom, and W. Cheng, 'Design of vector quantization codebooks using a genetic algorithm,' Proc. of IEEE International Conf. on EVolutionary Computation, pp.525-529, 1997 https://doi.org/10.1109/ICEC.1997.592366
  20. http://www.ics.uci.edu/~mlearn/databases/
  21. Yifeng Zhang and Siddhartha Bhattacharyya, 'Genetic progranuning in classifying large-scale data: an ensemble method,' Information Sciences, Vol.163, pp.85-101, 2004 https://doi.org/10.1016/j.ins.2003.03.028