한국정보통신학회논문지 (Journal of the Korea Institute of Information and Communication Engineering)

  • 제26권2호
  • /
  • Pages.325-328
  • /
  • 2022
  • /
  • 2234-4772(pISSN)
  • /
  • 2288-4165(eISSN)
한국정보통신학회 (The Korea Institute of Information and Commucation Engineering)
권호 표지
DOI QR코드

DOI QR Code

CNN에서 입력 최댓값을 이용한 SoftMax 연산 기법

SoftMax Computation in CNN Using Input Maximum Value

  • Kang, Hyeong-Ju (School of Computer Science and Engineering, Korea University of Technology and Education)
  • 투고 : 2021.12.21
  • 심사 : 2022.01.22
  • 발행 : 2022.02.28
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

A convolutional neural network(CNN) is widely used in the computer vision tasks, but its computing power requirement needs a design of a special circuit. Most of the computations in a CNN can be implemented efficiently in a digital circuit, but the SoftMax layer has operations unsuitable for circuit implementation, which are exponential and logarithmic functions. This paper proposes a new method to integrate the exponential and logarithmic tables of the conventional circuits into a single table. The proposed structure accesses a look-up table (LUT) only with a few maximum values, and the LUT has the result value directly. Our proposed method significantly reduces the space complexity of the SoftMax layer circuit implementation. But our resulting circuit is comparable to the original baseline with small degradation in precision.

키워드

과제정보

This work was also supported by the 2020 Professor Education and Research Promotion Program of KOREATECH, also supported by IDEC (EDA Tool), and also supported by "Regional Innovation Strategy (RIS)" through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(MOE)(2021RIS-004).

참고문헌

  1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proceedings of Advances in Neural Information Processing Systems, Lake Tahoe: NV, pp. 1097-1105, 2012.
  2. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proceedings of International Conference on Learning Representations, San Diego: CA, pp. 1-14, 2015.
  3. J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Sog, S. Guadarrama, and K. Murphy, "Speed/accuracy trade-offs for modern convolutional object detectors," in Proceedings of Computer Vision and Pattern Recognition, Honolulu: HW, pp. 1-21, 2017.
  4. M. Tan and Q. V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in Proceedings of International Conference on Machine Learning, Long Beach: CA, pp. 6105-6114, 2019.
  5. M. Tan, R. Pang, and Q. V. Le, "EfficientDet: Scalable and Efficient Object Detection," in Proceedings of Computer Vision and Pattern Recognition, online, pp. 10781-10790, 2020.
  6. Y. J. Kim and E. G. Kim, "Image based Fire Detection using Convolutional Neural Network," Journal of the Korea Institute of Information and Communication Engineering, vol. 20, no. 9, pp. 1649-1656, Sep. 2016. https://doi.org/10.6109/JKIICE.2016.20.9.1649
  7. B. Yuan, "Efficient hardware architecture of softmax layer in deep neural network," in Proceedings of IEEE International System-on-Chip Conference (SOCC), Seattle: WA, pp. 323-326, Sep. 2016.
  8. J. Park and D. Jeon, "Designing neuromorphic processor with on-chip learning," IDEC Journal of Integrated Circuits and Systems, vol. 6, no. 2, pp. 1-6, Apr. 2020. https://doi.org/10.23075/JICAS.2020.6.2.001