The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of the Effect on the Quantization of the Network's Outputs in the Neural Processor by the Implementation of Hybrid VLSI

하이브리드 VLSI 신경망 프로세서에서의 양자화에 따른 영향 분석

  • 권오준 (동의대학교 컴퓨터.영상공학부) ;
  • 김성우 (동의대학교 컴퓨터.영상공학부) ;
  • 이종민 (동의대학교 컴퓨터.영상공학부)
  • Published : 2002.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

In order to apply the artificial neural network to the practical application, it is needed to implement it with the hardware system. It is most promising to make it with the hybrid VLSI among various possible technologies. When we Implement a trained network into the hybrid neuro-chips, it is to be performed the process of the quantization on its neuron outputs and its weights. Unfortunately this process cause the network's outputs to be distorted from the original trained outputs. In this paper we analysed in detail the statistical characteristics of the distortion. The analysis implies that the network is to be trained using the normalized input patterns and finally into the solution with the small weights to reduce the distortion of the network's outputs. We performed the experiment on an application in the time series prediction area to investigate the effectiveness of the results of the analysis. The experiment showed that the network by our method has more smaller distortion compared with the regular network.

인공 신경망을 실제적인 응용 분야에 적용하기 위하여 하드웨어 시스템으로 구현하는 것이 필요하다. 하드웨어로 구현하는 방법에는 현재 하이브리드 VLSI 신경망 칩으로 구현하는 것이 가장 유망하다. 이미 학습된 신경망을 하이브리드 신경망 칩을 사용하여 구현하는 경우 뉴런 출력과 가중치 값의 양자화 과정이 필수적이다. 이러한 과정은 신경망의 출력층 뉴런의 이미 학습된 출력에 비해 왜곡을 야기한다. 본 논문에서는 이러한 신경망의 출력 왜곡에 대한 통계적 특성을 자세하게 분석하였다. 분석 결과는 신경망의 출력 왜곡을 줄이기 위해서는 입력 벡터의 정규화와 가중치 값들이 작아야 한다는 사실을 보여 주었다. 시계열 데이터에 대한 실험 결과는 분석 결과를 고려하여 학습된 신경망들의 경우 실제로 뉴런 출력 및 가중치 값의 양자화로 인한 출력층 뉴런의 출력 왜곡이 상당히 줄어들 수 있음을 명확히 보여 주었다.

Keywords

References

  1. B. Widrow, D. E. Rumelhart, and M. A. Lehr, 'Neural Networks : Applications in industry, business and science,' Communications of the ACM, Vol.37, No.3, pp.93-105, Mar., 1994 https://doi.org/10.1145/175247.175257
  2. 포항공대, '신경망 칩 으용 기반 기술 연구', 한국통신(KT-94-45) 장기 기초 연구 과제 보고서, Dec., 1993
  3. E. Sackinger, B. E. Boser, J. Bromley, Y. LeCun, and L. D. Jackel, 'Application of the ANNA Neural Network Chip to High Speed Character Recognition,' IEEE Transactions on Neural Networks, Vol.3, No.3, pp.498-505,1992 https://doi.org/10.1109/72.129422
  4. Maryhelen Stevenson, Rodney Winter and Bernard Widrow, 'Sensitivity of Feedforward Neural Networks to Weight Errors,' IEEE Trans, on Neural Networks, Vol.1, pp.71-80, 1990 https://doi.org/10.1109/72.80206
  5. Yun Xie and Marwan A. Jabri, 'Analysis of the Effects of Quantization in Multilayer Neural Networks Using a Statistical Model,' IEEE Trans, on Neural Networks, Vol.3, pp.334-338, 1992 https://doi.org/10.1109/72.125876
  6. Stephen W. Pich e, 'The Selection of Weight Accuracies for Madaline,' IEEE Trans, on Neural Networks, Vol.6, pp. 432-445, 1995 https://doi.org/10.1109/72.363478
  7. Jin-Young Choi and Chong-Ho Choi, 'Sensitivity Analysis of Multilayer Perceptron with Differentiable Activation Functions,' IEEE Trans, on Neural Networks, Vol.3, pp. 101-107, 1992 https://doi.org/10.1109/72.105422
  8. D. Lovell, P. Bartlett and T. Downs, 'Error and Variance Bounds on Sigmoidal Neurons with Weight and Input Errors,' Electronics Letters, Vol.28, pp.760-762, 1992 https://doi.org/10.1049/el:19920480
  9. Jordan L. Holt and Jenq-Neng Hwang, 'Finite Precision Error Analysis of Neural Network Hardware Implementations,' IEEE Trans, on Computers, Vol.42, pp.281-290, 1993 https://doi.org/10.1109/12.210171
  10. A. Papoulis, Probability, Random Variables, and Stochastic Processes, McGraw-Hill, 1987
  11. Oh-Jun Kwon and Sung-Yang Bang, 'Design of a Fault Tolerant Neural Network with a Desired Level of Robustness,' Electronics Letters, Vol.33, No.12, pp.1055-1057, 1997 https://doi.org/10.1049/el:19970729
  12. Neil A. Gershenfeld and Andreas S. Weigend, 'The Future of Time Series : Learning and Understanding,' In Time Series Prediction : Forecasting the Future and Understanding the Past, Addison Wesley, 1993
  13. Udo ubner, Carl-Otto Weiss, Neal Broadus Abraham, and Dingyuan Tang, 'Lorenz-Like Chaos in $NH_3-FIR$ Lasers(Data Set A),' In Time Series Prediction : Forecasting the Future and Understanding the Past, Addison Wesley, 1993