Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지

On-chip Learning Algorithm in Stochastic Pulse Neural Network

확률 펄스 신경회로망의 On-chip 학습 알고리즘

  • 김응수 (대전대학교 공과대학 전자공학과) ;
  • 조덕연 (선문대학교 대학원) ;
  • 박태진 (선문대학교 대학원)
  • Published : 2000.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper describes the on-chip learning algorithm of neural networks using the stochastic pulse arithmetic. Stochastic pulse arithmetic is the computation using the numbers represented by the probability of 1' and 0's occurrences in a random pulse stream. This stochastic arithmetic has the merits when applied to neural network ; reduction of the area of the implemented hardware and getting a global solution escaping from local minima by virtue of the stochastic characteristics. And in this study, the on-chip learning algorithm is derived from the backpropagation algorithm for effective hardware implementation. We simulate the nonlinear separation problem of the some character patterns to verify the proposed learning algorithm. We also had good results after applying this algorithm to recognize printed and handwritten numbers.

본 논문은 확률 펄스연산을 이용한 신경회로망이 on-Chip학습 알고리즘에 대해 기술하였다. 확률 펄스 연산은 임이의 펄스열에서 1과 0이 발생할 확률을 통해 표현된 수를 사용하여 계산하는 것을 일컫는다. 이러한 확률연산을 신경회로망에 적용하면 하드웨어 구현먼적을 줄일 수 있다는 것과 확률적인 특징으로 인해 지역 최소값으로부터 빠져 나와 광역 최적해에 도달할 수 있다는 장점을 갖고 있다. 또한 본 연구에서는 칩 냅에 학습할 수 있는 on-chip학습 알고리즘을 역전파 학습 알고리즘으로부터 유도하였다. 이렇게 유도된 알고리즘을 검증하기 위하여 비선형 패턴분리문제를 모의실험 하였다. 도한 활자체 및 필기체 숫자 인식에도 적용하여 좋은 결과를 얻었다.

Keywords

References

  1. Stochastic computing system B.R. Gaines
  2. IEEE Trans. On Neural Networks v.10 no.3 Frequency-Based Multilayer Neural Network with On-Chip Learning and Enhanced Neuron Characteristics Hiroomi Hikawa
  3. Digital VLSI for Neural Networks Dan Hammerstrom
  4. Analog VLSI for Neural Networks Robert W. Newcomb;Jason D. Lohn
  5. IEEE Trans. on Neural Networks v.3 no.3 Functional Abilities of a Stochastic Logic Neural Network Y. Kondo;Y. Sawada
  6. Parallel Distributed Processing v.1 no.ch. 8 Learning internal representations by error propagation D.E. Rumelhart;G.E. Hinton;R.J. Williams
  7. Neural Networks for Optimization and Signal Processing A. Cichoki;R. Unbehauen
  8. Proc. Of IJCNN v.2 Frequency based computation in neural networks G.S. Eisman
  9. DARPA Neural Network Study Final Report
  10. Analogue neural VLSI a pulse stream approach A. Murray;L. Tarassenko
  11. IEEE J. Solid-State Circuits v.25 Pulse-density modulation technique in VLSI implementations of neural network algorithms J.E. Tomberg;K.K.K. Kaski
  12. IEEE Trans. On CAD v.8 no.8 Cellular Automata-Based Pseudorandom Number Generators for Built-In-Self-Test P.D. Hartensius;R.D. McLeod;Werner Pries;D. Michael Miller;H.C. Card
  13. IEEE Trans. On Circuits and Systems v.38 no.1 A VLSI-Efficient Technique for Generating Multiple Uncorrelated Noise Sources and Its Application to Stochastic Neural Networks J. Apspector;J.W. Gannett;S. Haber;Michael B. Parker;Robert Chu
  14. Proc. Of IJCNN v.1 Neural Network LSI Chip with On-chip Learning H. Eguchi;R. Furuta;H. Horigushi;S. Oteki;T. Kitabuchi
  15. Proc. Of IJCNN v.2 digital implementation issues of stochastic neural networks E.E. Pesulima;A.S. Pandya;R. Shankar
  16. IEEE Trans. On Circuits and Systems v.38 Analysis of Gradient Descent Learning Algorithm for Multilayer Feedforward Neural Networks H. Guo;S.B. Gelfand
  17. Int'l Conf. On Info., Comm. and Signal Processing A Stochastic Backpropagation Algorithm for Training Neural Networks Y.Q. Chen;T. Yin;H.A. Babri
  18. IEEE Trans. On Signal Processing v.44 no.8 Fully Parallel Stochastic Computation Architecture C.L. Janer;J.M. Quero;J.G. Ortega;L.G. Franquelo
  19. Stochastic pulse Coding을 이용한 신경회로망의 구현 서정원
  20. 펄스열에서 1인 펄스수와 0인 펄스수의 비를 이용한 확률연산에 관한 연구 민승재;이일완;채수익
  21. Analog VLSI for Neural Networks Robert W. Newcomb;Jason D. Lohn