Convergence Security Journal (융합보안논문지)

Korea convergence Security Association (한국융합보안학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-type object detection-based de-identification technique for personal information protection

개인정보보호를 위한 다중 유형 객체 탐지 기반 비식별화 기법

  • 길예슬 (성신여자대학교 미래융합기술공학과) ;
  • 이효진 (성신여자대학교 미래융합기술공학과) ;
  • 류정화 (성신여자대학교 융합보안공학과) ;
  • 이일구 (성신여자대학교 미래융합기술공학과)
  • Received : 2022.10.01
  • Accepted : 2022.11.01
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the Internet and web technology develop around mobile devices, image data contains various types of sensitive information such as people, text, and space. In addition to these characteristics, as the use of SNS increases, the amount of damage caused by exposure and abuse of personal information online is increasing. However, research on de-identification technology based on multi-type object detection for personal information protection is insufficient. Therefore, this paper proposes an artificial intelligence model that detects and de-identifies multiple types of objects using existing single-type object detection models in parallel. Through cutmix, an image in which person and text objects exist together are created and composed of training data, and detection and de-identification of objects with different characteristics of person and text was performed. The proposed model achieves a precision of 0.724 and mAP@.5 of 0.745 when two objects are present at the same time. In addition, after de-identification, mAP@.5 was 0.224 for all objects, showing a decrease of 0.4 or more.

인터넷과 웹 기술이 모바일 장치 중심으로 발전하면서 이미지 데이터는 사람, 텍스트, 공간 등 다양한 유형의 민감정보를 담고 있다. 이러한 특성과 더불어 SNS 사용이 증가하면서 온라인 상의 개인정보가 노출되고 악용되는 피해 규모가 커지고 있다. 그러나 개인정보보호를 위한 다중 유형 객체 탐지 기반의 비식별화 기술에 관한 연구는 미흡한 상황이다. 이에 본 논문은 기존의 단일 유형 객체 탐지 모델을 병렬적으로 이용하여 다중 유형의 객체를 탐지 및 비식별화하는 인공지능 모델을 제안한다. Cutmix 기법을 통해 사람과 텍스트 객체가 함께 존재하는 이미지를 생성하여 학습 데이터로 구성하고, 사람과 텍스트라는 다른 특징을 가진 객체에 대한 탐지 및 비식별화를 수행하였다. 제안하는 모델은 두 가지 객체가 동시에 존재할 때 0.724의 precision과 0.745의 mAP@.5 를 달성한다. 또한, 비식별화 수행 후 전체 객체에 대해 mAP@.5 가 0.224로, 0.4 이상의 감소폭을 보였다.

Keywords

Acknowledgement

이 논문은 2022년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원(No. 2020R1F1A1061107)과 2022년도 정부(산업통상자원부)의 재원으로 한국산업기술진흥원의 지원(P0008703, 2022년 산업혁신인재성장지원사업), 과학기술정보통신부 및 정보통신기획평가원의 ICT혁신인재4.0 사업의 연구결과로 수행되었음 (IITP-2022-RS-2022-00156310).

References

  1. Jain, A.K., Sahoo, S.R. & Kaubiyal, J. "Online social networks security and privacy: comprehensive review and analysis". Complex Intell. Syst. 7, 2157-2177 (2021) https://doi.org/10.1007/s40747-021-00409-7
  2. M. Carranza-Garcia , J. Torres-Mateo, P. Lara-Benitez and J. Garcia-Gutierrez, "On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data", Remote Sens. 2021, vol. 13, no. 1, pp. 89, 2020.
  3. L. Jiao, F. Zhang, F. Liu, S. Yang, L. Li, Z. Feng and R. Qu., "A Survey of Deep Learning-based Object Detection", arXiv:1907.09408v2, 2019. https://doi.org/10.1109/ACCESS.2019.2939201
  4. H. Chen, Z. He, B. Shi, and T. Zhong, "Research on Recognition Method of Electrical Components Based on YOLO V3", IEEE Access, vol. 7, pp. 157818 - 157829, 2019.
  5. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", arXiv:1506.02640v5, 2015.
  6. J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement", arXiv:1804.02767, 2018.
  7. Y. Du, C. Li, R. Guo, X. Yin, W. Liu, J. Zhou, Y. Bai, Z. Yu, Y. Yang, Q. Dang and H. Wang, "PP-OCR: A Practical Ultra Lightweight OCR System", arXiv:2009.09941, 2020.
  8. Bhujbal, Avinash and Deepak T. Mane. "A Survey On Deep Learning Approaches For Vehicle And Number Plate Detection." International Journal of Scientific & Technology Research 8 (2019): 1378-1383.
  9. Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, "Character Region Awareness for Text Detection", arXiv:1904.01941v1, 2019.
  10. S. Yun, D. Han, S. Oh, S. Chun, J. Choe and Y. Yoo, "CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features", arXiv:1905.04899, 2019.
  11. SR. Klomp, M. Rijn, R. Wijnhoven, C. Snoek and P. De With, "Safe Fakes: Evaluating Face Anonymizers for Face Detectors", Proceedings of 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021.
  12. H. Hukkelas, R. Mester. and F. Lindseth, "DeepPrivacy: A Generative Adversarial Network for Face Anonymization", arXiv:1909.04538, 2019.
  13. C. Liu, T. Zhu, J. Zhang and W. Zhou, "Privacy Intelligence: A Survey on Image Privacy in Online Social Networks", arXiv:2008.12199v2, 2020.
  14. L. Schnabel, S. Matzka, M. Stellmacher, M. Patzold and E. Matthes, "Impact of Anonymization on Vehicle Detector Performance" Proceedings of 2019 Second International Conference on Artificial Intelligence for Industries (AI4I), 2019.
  15. L. Du. and H. Ling., "Preservative License Plate De-identification for Privacy Protection" Proceedings of 2011 International Conference on Document Analysis and Recognition, 2011.
  16. J. Yu, H. Xue, B. Liu, Y. Wang, S. Zhu and M. Ding "GAN_based differential private image privacy protection framework" Sensors 2021, vol. 21, no. 1, pp. 58, 2020.
  17. Z. Ge, S. Liu, F. Wang, Z. Li and J. Sun, "YOLOX: Exceeding YOLO Series in 2021", arXiv:2107.08430, 2021.
  18. Nanonets, "How to OCR with Tesseract, OpenCV and Python". Nanonets, July 2022, https://nanonets.com/blog/ocr-with-tesseract/, accessed 30 Sep 2022.
  19. L. Wang, J. Shi and G. Song& I. Shen, "Object detection combining recognition and segmentation.", Asian conference on computer vision, Springer, Berlin, Heidelberg, 2007.
  20. N. J. Karthika, and S. Chandran, "Addressing the False Positives in Pedestrian Detection.", Electronic Systems and Intelligent Computing, Springer, Singapore, 2020. 1083-1092.
  21. A. Singh, G. Pang, M. Toh, J. Huang, W. Galuba and T. Hassner, "TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text.", Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
  22. TY. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, L. Zitnick and P. Dollar "Microsoft coco: Common objects in context.", European conference on computer vision, Springer, Cham, 2014.
  23. A. Veit, T. Matera, L. Nemann, J. Matas and S. Belongie, "Coco-text: Dataset and benchmark for text detection and recognition in natural images.", arXiv:1601.07140, 2016.
  24. Zhu, Mu. "Recall, precision and average precision." Department of Statistics and Actuarial Science, University of Waterloo, Waterloo 2.30 (2004): 6.