전기학회논문지 (The Transactions of The Korean Institute of Electrical Engineers)

  • 제67권12호
  • /
  • Pages.1665-1670
  • /
  • 2018
  • /
  • 1975-8359(pISSN)
  • /
  • 2287-4364(eISSN)
대한전기학회 (The Korean Institute of Electrical Engineers)
권호 표지
DOI QR코드

DOI QR Code

진화연산 기반 CNN 필터 축소

Evolutionary Computation Based CNN Filter Reduction

  • Seo, Kisung (Department of Electronics Engineering, Seokyeong University)
  • 투고 : 2018.11.01
  • 심사 : 2018.11.29
  • 발행 : 2018.12.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

A convolutional neural network (CNN), which is one of the deep learning models, has been very successful in a variety of computer vision tasks. Filters of a CNN are automatically generated, however, they can be further optimized since there exist the possibility of existing redundant and less important features. Therefore, the aim of this paper is a filter reduction to accelerate and compress CNN models. Evolutionary algorithms is adopted to remove the unnecessary filters in order to minimize the parameters of CNN networks while maintaining a good performance of classification. We demonstrate the proposed filter reduction methods performing experiments on CIFAR10 data based on the classification performance. The comparison for three approaches is analysed and the outlook for the potential next steps is suggested.

키워드

참고문헌

  1. J. Schmidhuber, "Deep Learning in Neural Networks: An Overview", Neural Networks, Vol. 61, pp. 85-117, 2015. https://doi.org/10.1016/j.neunet.2014.09.003
  2. Y. LeCun, Y. Bengio, G. Hinton, "Deep learning," Nature, Vol. 521, pp. 436-444, 2015. https://doi.org/10.1038/nature14539
  3. LeCun, Yann, et al. "Gradient based learning applied to document recognition", Proceedings of the IEEE, pp. 2278-2324, 1998.
  4. Yunwon Park, In So Kweon "Ambiguous Surface Defect Image Classification of AMOLEDD is playsin Smartphones", IEEE Trans. Industrial Informatics, 12(2): 597-607,2016 https://doi.org/10.1109/TII.2016.2522191
  5. K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", International Conference on Learning Representations, 2014.
  6. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going Deeper with Convolutions", Computer Vision and Pattern Recognition, 2015
  7. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition", Computer Vision and Pattern Recognition, 2016..
  8. S. Zagoruyko and N. Komodakis, "Wide Residual Networks", arXiv: 1605.07146, 2016.
  9. C. Fernando et al. "Convolution by Evolution: Differentiable Pattern Producing Networks", In Proceedings of the 2016 Genetic and Evolutionary Computation Conference, Denver, CO, USA, pp. 109-116. 2016.
  10. A. Rikhtegar, M. Pooyan, M. Manzuri-Shalmani, "Genetic algorithm-optimised structure of convolutional neural network for face recognition applications", IET Computer Vision, Vol. 10, Iss. 6, pp. 559-566, 2016 https://doi.org/10.1049/iet-cvi.2015.0037
  11. L. Xie, A. Yuille, "Genetic CNN", CVPR 2017
  12. M. Suganuma, s, Shirakawa, T. Nagao, "A Genetic Programming Approach to Designing Convolutional Neural Network Architectures", Proceedings of GECCO 2017, pp. 497-504, 2017
  13. B. Zoph, and Q. V. Le, "Neural Architecture Search with Reinforcement Learning", CoRR abs/1611.01578 2016
  14. C. Liu, B. Zoph, J. Shlens, W. Hua, L. J. Li, L. Fei-Fei, and K. Murphy, "Progressive Neural Architecture Search", ECCV 2018
  15. E. Real, A. Aggarwal, Y. Huang, Q. V. Le, "Aging Evolution for Image Classifier Architecture Search", 2018
  16. H. Hu, R. Peng, Y.W. Tai, and C.K. Tang. "Network trimming: A data-driven neuron pruning approach towards efficient deep architectures", arXiv preprint arXiv:1607.03250, 2016.
  17. H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning Filters for Efficient ConvNets", CoRR abs/1608.08710 (2016).
  18. Q. Huang, K. Zhou, S. You, and U. Neumann, "Learning to prune filters in convolutional neural networks", arXiv preprint arXiv:1801.07365. 2018 Jan 23.
  19. C. Chen, F. Tung, N. Vedula, and G. Mori, "Constraint-Aware Deep Neural Network Compression", ECCV (8) 2018: 409-424.
  20. S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding", arXiv preprint arXiv:1510.00149, 2015.
  21. J. H. Luo, J. Wu, and W. Lin, "ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression", ICCV 2017: 5068-5076.
  22. J. Kim, M. Lee, J. Choi, K. Seo, "GA-based Filter Selection for Representation in Convolutional Neural Networks", ECCV 2018 Workshop on Compact and Efficient Feature Representation and Learning in Computer Vision.
  23. K. Seo, "Analysis of evolutionary optimization methods for CNN structures", Transactions of the Korean Institute of Electrical Engineers, 67(6), pp. 767-772, 2018 https://doi.org/10.5370/KIEE.2018.67.6.767