The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of Neural Networks using GPU

GPU를 이용한 신경망 구현

  • Published : 2004.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

We present a new use of common graphics hardware to perform a faster artificial neural network. And we examine the use of GPU enhances the time performance of the image processing system using neural network, In the case of parallel computation of multiple input sets, the vector-matrix products become matrix-matrix multiplications. As a result, we can fully utilize the parallelism of GPU. Sigmoid operation and bias term addition are also implemented using pixel shader on GPU. Our preliminary result shows a performance enhancement of about thirty times faster using ATI RADEON 9800 XT board.

본 논문은 일반적인 그래픽스 하드웨어를 이용하여 더욱 빠른 신경망을 구현하고, 구현된 시스템을 영상 처리 분야에 적용함으로써 효용성을 검증한다. GPU의 병렬성을 효율적으로 사용하기 위하여, 다수의 입력벡터와 연결가중치벡터를 모아서 많은 내적연산을 하나의 행렬곱 연산으로 대체하였고, 시그모이드와 바이어스 항 덧셈 연산도 GPV 상에서 픽셀세이더로 구현하였다. ATI RADEON 9800 XT 보드를 이용하여 구현된 신경망 시스템은 CPU를 사용한 기존의 시스템과 비교하여 정확도의 차이 없이 30배 정도의 속도 향상을 얻을 수 있었다.

Keywords

References

  1. K. Oh, B. Shin and Y. G. Shin, 'Mobility Culling-An Efficient Rendering Algorithm Using Temporal Coherence,' The Journal of Visualization and Computer Animation, Vol.12, Issue 3, pp.159-166, 2001 https://doi.org/10.1002/vis.253
  2. E. S. Larsen and D. McAllister, 'Fast Matrix Multiplies using Graphics Hardware,' Proceedings of the 2001 ACM/IEEE Conference on Supercomputing, pp.55-55, 2001 https://doi.org/10.1145/582034.582089
  3. J. D. Hall, N. A. Carr and J. C. Hart, 'Cache and Bandwidth Aware Matrix Multiplication on the GPU,' Technical Report UIUCDCS-R-2003-2328, University of Illinois Dept. of Computer Science, Mar., 2003
  4. A. Moravanszky, 'Linear Algebra on the GPU,' in : W. F. Engel (Ed), Shader X2, Wordware Publishing, 2003
  5. J. Kruger and R. Westermann, 'Linear Algebra Operators for GPU Implementation of Numerical Algorithms,' SIG-GRAPH 2003, 2003 https://doi.org/10.1145/1201775.882363
  6. R. Yang and G. Welch. 'Fast Image Segmentation and Smoothing Using Commodity Graphics Hardware,' the Journal of Graphics Tools, Vol.7, No.4, pp.91-100, 2003 https://doi.org/10.1080/10867651.2002.10487576
  7. D. Manocha, 'Interactive Geometric & Scientific Computations using Graphics Hardware,' SIGGRAPH 2003 Tutorial Course #11, 2003
  8. M. M. Trentacoste, 'Implementing Performance Libraries on Graphics Hardware,' Carnegie Mellon University Undergraduate Thesis, 2003
  9. J. Zhu and P. Sutton, 'FPGA Implementation of Neural Networks - a Survey of a Decade of Progress,' Proceedings of 13th International Conference on Field Programmable Logic and Applications (FPL 2003), 2003
  10. Haykin, Neural Networks, Prentice Hall, 1999
  11. Biebelmann, E., Koppen, M. and Nickolay, B., 'Practical Applications of Neural Networks in Texture Analysis,' Neurocomputing, Vol.13, pp.261-279, 1996 https://doi.org/10.1016/0925-2312(95)00092-5
  12. H. Li, D. Doerman and O. Kia, 'Automatic Text Detection and Tracking in Digital Video,' IEEE Transactions on Image Processing, Vol.9, No.1, pp.147-156, 2000 https://doi.org/10.1109/83.817607
  13. Y. Zhong, H. Zhang and A. K. Jain, 'Automatic Caption Localization in Compressed Video,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.4, 2000 https://doi.org/10.1109/ICIP.1999.822862
  14. K. Jung, K. I. Kim and A. K. Jain, 'Text Information Extraction in Images and Video: A Survey,' International Journal of Pattern Recognition, Vol.37, Issue 5, pp.977-997, May, 2004 https://doi.org/10.1016/j.patcog.2003.10.012
  15. K. Jung, 'Neural network-based Text Location in Color Images,' Pattern Recognition Letters, Vol.22, No.14, pp.1503-1515, 2001 https://doi.org/10.1016/S0167-8655(01)00096-4
  16. K. Y. Jeong, K. Jung, E. Y. Kim and H. J. Kim, 'Neural Network-based Text Location for News Video Indexing,' Proceedings of International Conference of Image Processing, 1999 https://doi.org/10.1109/ICIP.1999.817127
  17. A. K. Jain, B. Yu, 'Automatic Text Location in Images and Video Frames,' Pattern Recognition, Vol. 31, No. 12, pp. 2055-2076, 1998 https://doi.org/10.1016/S0031-3203(98)00067-3
  18. Antonio d'Acierno, 'Back-Propagation Learning Algorithm and Parallel Computers: The CLEPSYDRA Mapping Scheme,' Neurocomputing, Vol.31, pp.67-85, 2000 https://doi.org/10.1016/S0925-2312(99)00151-4
  19. Nazeih M. Botros and M. Abdul-Aziz, 'Hardware Implementation of an Artificial Neural Network Using Field programmable Gate Array(FPGA's)' IEEE Transactions on Industrial Electronics, Vol.41, No.6, December, 1994 https://doi.org/10.1109/41.334585
  20. G. -P. K. Economou, E. P. Mariatos, N. M. Economopoulos, D. Lymberopoulos and C. E. Goutis, 'FPGA Implementation of Artificial Neural Networks : An Application on Medical Expert Systems,' 4th International Conference on Microelectronics for Neural Networks and Fuzzy Systems, Torino, Italy, pp.287-293, September, 1994 https://doi.org/10.1109/ICMNN.1994.593722