Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Parameter Optimization of Extreme Learning Machine Using Bacterial Foraging Algorithm

Bacterial Foraging Algorithm을 이용한 Extreme Learning Machine의 파라미터 최적화

  • 조재훈 (충북대학교 전기전자컴퓨터공학부) ;
  • 이대종 (충북대학교 BK21 충북정보기술사업단) ;
  • 전명근 (충북대학교 전기전자컴퓨터공학부)
  • Published : 2007.12.25
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, Extreme learning machine(ELM), a novel learning algorithm which is much faster than conventional gradient-based learning algorithm, was proposed for single-hidden-layer feedforward neural networks. The initial input weights and hidden biases of ELM are usually randomly chosen, and the output weights are analytically determined by using Moore-Penrose(MP) generalized inverse. But it has the difficulties to choose initial input weights and hidden biases. In this paper, an advanced method using the bacterial foraging algorithm to adjust the input weights and hidden biases is proposed. Experiment at results show that this method can achieve better performance for problems having higher dimension than others.

최근 단일 은닉층을 갖는 전방향 신경회로망 구조로, 기존의 경사 기반 학습알고리즘들보다 학습 속도가 매우 우수한 ELM(Extreme Learning Machine)이 제안되었다. ELM 알고리즘은 입력 가중치들과 은닉 바이어스들의 초기 값을 무작위로 선택하고 출력 가중치들은 Moore-Penrose(MP) 일반화된 역행렬 방법을 통하여 구해진다. 그러나 입력 가중치들과 은닉층 바이어스들의 초기 값 선택이 어렵다는 단점을 갖고 있다. 본 논문에서는 최적화 알고리즘 중 박테리아 생존(Bacterial Foraging) 알고리즘의 수정된 구조를 이용하여 ELM의 초기 입력 가중치들과 은닉층 바이어스들을 선택하는 개선된 방법을 제안하였다. 실험을 통하여 제안된 알고리즘이 많은 입력 데이터를 가지는 문제들에 대하여 성능이 우수함을 보였다.

Keywords

References

  1. Richard O. Duda, Peter E. Hart, David G. Stork, Pattern Classification, John Wiley & Sons, 2001
  2. J-S. R. Jang, C. T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing : A Computational Approach to Learning and Machine Intelligence, Prentice Hall, 1997
  3. J. S. R. Jang, 'ANFIS : Adaptive Network-based Fuzzy Inference System,' IEEE Trans on System, Man, and Cybernetics, Vol. 23, No.3, pp. 665- 685, 1993 https://doi.org/10.1109/21.256541
  4. Mohammad Fazle Azeem, Madasu Hanmandlu, and Nesar Ahmad, 'Structure Identification of Generalized Adaptive Neuro-Fuzzy Inference Systems,' IEEE Trans on Fuzzy Systems, Vol. 11, No.5, pp. 666-681. 2003 https://doi.org/10.1109/TFUZZ.2003.817857
  5. D. Rumelhart, and J. McClelland, 'Parallel distributed processing', Vol. 1, MIT Press, 1986
  6. S. Suresh, S.N. Omkar, and V. Mani, 'Parallel implementation of back-propagation algorithm in networks of workstations', IEEE Transactions on Parallel and Distributed Systems, Vol. 16, pp. 24-34, 2005 https://doi.org/10.1109/TPDS.2005.11
  7. Hsu, C.-T. Kang, M.-S. and Chen, C.-S, 'Design of adaptive load shedding by artificial neural networks', IEE Proceed., Generation, Transmission and Distribution, Vol. 152, pp. 415-421, 2005 https://doi.org/10.1049/ip-gtd:20041207
  8. G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, 'Extreme learning machine: a new learning scheme of feedforward neural networks', International Joint Conference on Neural Networks (IJCNN2004), pp. 25-29, 2004
  9. M.-B. Li, G.-B. Huang, P. Saratchandran, N. Sundararajan, 'Fully complex extreme learning machine', Neurocomputing, vol. 68, pp. 306-314, 2005 https://doi.org/10.1016/j.neucom.2005.03.002
  10. Qin-Yu Zhu, A.K. Qin, P.N. Suganthan, Guang-Bin Huang, 'Evolutionary extreme learning machine', Pattern Recognition, Vol. 38, pp. 1759-1763, 2005 https://doi.org/10.1016/j.patcog.2005.03.028
  11. 한승수, '유전알고리즘을 이용한 신경망 구조 및 파라미터 최적화', 퍼지 및 지능시스템학회 논문집, Vol. 11, No.3, pp. 215-222, 2001
  12. G.W. Game and C. D. James, 'The application of genetic algorithms to the optimal selection of parameter values in neural networks for attitude control system', IEEE Colloquium High Accuracy Platform Control Space, 1993
  13. H. J. Bremermann, 'Chemotaxis and optimization,' J. Franklin Inst., Vol. 297, pp. 397-404, 1974 https://doi.org/10.1016/0016-0032(74)90041-6
  14. PASSINO, K. M., 'Biomimicry of Bacterial foraging for Distributed Optimization', University Press, Princeton, New Jersey, 2001
  15. W. John O'brien, Howard I. Browman, and Barbara I. Evans, 'Search Strategies of Foraging Animals,' American Scientist, Vol. 78 pp. 152-160, 1990
  16. R. M. Tong, 'The Evaluation of Fuzzy Models Derived from Experimental Data,' Fuzzy Sets and Systems, Vol. 4, pp. 1-12, 1980 https://doi.org/10.1016/0165-0114(80)90059-7

Cited by

  1. Hybrid Bacterial Foraging and Particle Swarm Optimization for detecting Bundle Branch Block vol.4, pp.1, 2015, https://doi.org/10.1186/s40064-015-1240-z
  2. Optimization of Fuzzy Learning Machine by Using Particle Swarm Optimization vol.26, pp.1, 2016, https://doi.org/10.5391/JKIIS.2016.26.1.087
  3. Optimal parameters of an ELM-based interval type 2 fuzzy logic system: a hybrid learning algorithm pp.1433-3058, 2016, https://doi.org/10.1007/s00521-016-2503-5
  4. Metaheuristic-based extreme learning machines: a review of design formulations and applications pp.1868-808X, 2018, https://doi.org/10.1007/s13042-018-0833-6