방송공학회논문지 (Journal of Broadcast Engineering)

  • 제24권6호
  • /
  • Pages.1076-1084
  • /
  • 2019
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

미디어 아카이브 구축을 위한 등장인물, 사물 메타데이터 생성 시스템 구현

Implementation of Character and Object Metadata Generation System for Media Archive Construction

  • 조성만 (서울과학기술대학교 일반대학원 미디어IT공학과) ;
  • 이승주 (서울과학기술대학교 일반대학원 미디어IT공학과) ;
  • 이재현 (서울과학기술대학교 일반대학원 미디어IT공학과) ;
  • 박구만 (서울과학기술대학교 일반대학원 미디어IT공학과)
  • Cho, Sungman (Seoul National University of Science and Technology) ;
  • Lee, Seungju (Seoul National University of Science and Technology) ;
  • Lee, Jaehyeon (Seoul National University of Science and Technology) ;
  • Park, Gooman (Seoul National University of Science and Technology)
  • 투고 : 2019.07.04
  • 심사 : 2019.09.04
  • 발행 : 2019.11.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문에서는 딥러닝을 적용하여 미디어 내의 등장인물 및 사물을 인식, 메타데이터를 추출하고 이를 통해 아카이브를 구축하는 시스템을 개발하였다. 방송 분야에서 비디오, 오디오, 이미지, 텍스트 등의 멀티미디어 자료들을 디지털 컨텐츠로 전환하기 시작한지는 오래 되었지만, 아직 구축해야 할 자료들은 방대하게 남아있다. 따라서 딥러닝 기반의 메타데이터 생성 시스템을 구현하여 미디어 아카이브 구축에 소모되는 시간과 비용을 절약 할 수 있도록 하였다. 전체 시스템은 학습용 데이터 생성 모듈, 사물 인식 모듈, 등장인물 인식 모듈, API 서버의 네 가지 요소로 구성되어 있다. 미디어 내에서 등장인물 및 사물을 인식하여 메타데이터로 추출할 수 있도록 딥러닝 기술로 사물 인식 모듈, 얼굴 인식 모듈을 구현하였다. 딥러닝 신경망을 학습시키기 위한 데이터를 구축하기 용이하도록 학습용 데이터 생성 모듈을 별도로 설계하였으며 얼굴 인식, 사물 인식의 기능은 API 서버 형태로 구성하였다. 1500명의 인물, 80종의 사물 데이터를 사용하여 신경망을 학습시켰으며 등장인물 테스트 데이터에서 98%, 사물 데이터에서 42%의 정확도를 확인하였다.

In this paper, we introduced a system that extracts metadata by recognizing characters and objects in media using deep learning technology. In the field of broadcasting, multimedia contents such as video, audio, image, and text have been converted to digital contents for a long time, but the unconverted resources still remain vast. Building media archives requires a lot of manual work, which is time consuming and costly. Therefore, by implementing a deep learning-based metadata generation system, it is possible to save time and cost in constructing media archives. The whole system consists of four elements: training data generation module, object recognition module, character recognition module, and API server. The deep learning network module and the face recognition module are implemented to recognize characters and objects from the media and describe them as metadata. The training data generation module was designed separately to facilitate the construction of data for training neural network, and the functions of face recognition and object recognition were configured as an API server. We trained the two neural-networks using 1500 persons and 80 kinds of object data and confirmed that the accuracy is 98% in the character test data and 42% in the object data.

키워드

참고문헌

  1. I.Choi, H.Song, S.Lee, J.Yoo, "Facial Expression Classification Using Deep Convolutional Neural Network", Journal of Broadcast Engineering, Vol.22, No.2, March 2017.
  2. H.Jun, G.Hyun, K.Lim, W.Lee, H.Kim, "Big Data Preprocessing for Predicting Box Office Success", KISE Transactions on Computing Practices, Vol.20, No.12, pp.615-622, December 2014. https://doi.org/10.5626/KTCP.2014.20.12.615
  3. F Schroff, D Kalenichenko, J Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015.
  4. W.Liu, D.Anguelov, D.Erhan, C.Szegedy, S.Reed, "SSD: Single shot multibox detector", arxiv:1512.02325, 2015.
  5. Y.Wen, K.Zhang, Z.Li, Y.Qiao, "A discriminative feature learning approach for deep face recognition", In European Conference on Computer Vision (ECCV), pages 499-515, 2016.
  6. W.Liu, Y.Wen, Z.Yu, M.Li, B.Raj, L.Song, "SphereFace: Deep Hypersphere Embedding for Face Recognition", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
  7. J.Redmon and A.Farhadi, "You only look once: Unified, real-time object detection", In CVPR, 2016.
  8. R.Girshick, J.Donahue, T.darrell, J.malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", In CVPR, 2014.
  9. Y.Kim, J.Woo, J.Lee, J.Shin, "High-quality data collection for machine learning using block chain", Journal of the Korea Institute of Information and Communication Engineering, Vol.23, No.1, pp.13-19, Jan 2019. https://doi.org/10.6109/JKIICE.2019.23.1.13
  10. K.Zhang, Z.Zhang, Z.Li, Y.Qiao, "Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks", IEEE SIGANL PROCESSING LETTERS, Vol.23, No.10, October 2016
  11. W.Kiri, C.Claire, R.Seth, S.Stefan, "Constrained K-means Clustering with Backgroud Knowledge", Proceedings of the Eighteenth International Conference on Machine Learning, pp.574-584, 2001
  12. Szegedy C, Ioffe S, Vanhoucke V, "Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning". In: AAAI. vol. 4, pp. 12, 2017.
  13. Chris Biemann, "Chinese Whispers - an Efficient Graph Clustering Algorithm and its Application to Natural Language Processing Problems", TextGraphs-1 Proceedings of the First Workshop on Graph Based Methods for Natural Language Processing, pp.73-80, 2006.
  14. Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollar, "Microsoft COCO: Common Objects in Context", arxiv : 1405.0312, 2015.
  15. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J. and Zisserman, A., "The PASCAL Visual Object Classes (VOC) Challenge", International Journal of Computer Vision, 88(2), 303-338, 2010 https://doi.org/10.1007/s11263-009-0275-4
  16. C. J. C. Burges, "A tutorial on Support vector machines for pattern recognition", Data mining and knowledge discovery 2, pp. 121-167, 1998. https://doi.org/10.1023/A:1009715923555
  17. B. Moore, "Principal component analysis in linear systems: Controllability, observability, and model reduction", IEEE Transactions on Automatic Control, pp. 17-32, Feb 1981.
  18. L Maaten, G Hinton, "Visualizing data using t-SNE", Journal of Machine Learning Research 9, pp. 2579-2605, Nov 2008.
  19. Andrew G.Howard, M.Zhu, Bo Chen, D.Kalenichnko, W.Wang, T.Weyand, M.Andreetto, H.Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications", CoRR, abs/1704.04861, 2017.