ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Vision-based garbage dumping action detection for real-world surveillance platform

  • Yun, Kimin (SW.Contents Laboratory, Electronics and Telecommunications Research Institute) ;
  • Kwon, Yongjin (SW.Contents Laboratory, Electronics and Telecommunications Research Institute) ;
  • Oh, Sungchan (SW.Contents Laboratory, Electronics and Telecommunications Research Institute) ;
  • Moon, Jinyoung (SW.Contents Laboratory, Electronics and Telecommunications Research Institute) ;
  • Park, Jongyoul (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
  • Received : 2018.09.18
  • Accepted : 2019.04.28
  • Published : 2019.08.02
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a new framework for detecting the unauthorized dumping of garbage in real-world surveillance camera. Although several action/behavior recognition methods have been investigated, these studies are hardly applicable to real-world scenarios because they are mainly focused on well-refined datasets. Because the dumping actions in the real-world take a variety of forms, building a new method to disclose the actions instead of exploiting previous approaches is a better strategy. We detected the dumping action by the change in relation between a person and the object being held by them. To find the person-held object of indefinite form, we used a background subtraction algorithm and human joint estimation. The person-held object was then tracked and the relation model between the joints and objects was built. Finally, the dumping action was detected through the voting-based decision module. In the experiments, we show the effectiveness of the proposed method by testing on real-world videos containing various dumping actions. In addition, the proposed framework is implemented in a real-time monitoring system through a fast online algorithm.

Keywords

References

  1. W. Liu et al., SSD: Single shot multibox detector, in Proc. Eur. Conf. Comput. Vision (ECCV), Amsterdam, The Netherlands, Oct. 11-14, 2016, pp. 21-37.
  2. J. Redmon et al., You only look once: Unified, real‐time object detection, in Proc. Comput. Vision Pattern Recogn. (CVPR), Las Vegas, NV, USA, June 27-30, 2016, pp. 779-788.
  3. S. Ren et al., Faster R‐CNN: Towards real‐time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39 (2017), 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
  4. K. Simonyan and A. Zisserman, Two‐stream convolutional networks for action recognition in videos, in Proc. Adv. Nneural Inf. Process. Syst. (NIPS), Montreal, Canada, Dec. 8-13, 2014, pp. 567-576.
  5. D. Tran et al., Learning spatiotemporal features with 3D convolutional networks, in Proc. Int. Conf. Comput. Vision (ICCV), Santiago, Chile, Dec. 7-13, 2015, pp. 4489-4497.
  6. L. Wang et al., Temporal segment networks: Towards good practices for deep action recognition, in Proc. Eur. Conf. Comput. Vision (ECCV), Amsterdam, The Netherlands, Oct. 11-14, 2016, pp. 20-36.
  7. F. Porikli, Y. Ivanov, and T. Haga, Robust abandoned object detection using dual foregrounds, EURASIP J. Adv. Signal Process. 30 (2008), 1-11.
  8. R.H. Evangelio and T. Sikora. Complementary background models for the detection of static and moving objects in crowded environments, in Proc. Adv. Video Signal‐Based Surveillance (AVSS), Klagenfurt, Austria, 2011, pp. 71-76.
  9. S. Kim et al., Intelligent visual surveillance‐A survey, Int. J. Contr. Autom. Syst. 8 (2010), 926-939. https://doi.org/10.1007/s12555-010-0501-4
  10. G. Chunhui et al., AVA: A video dataset of spatio‐temporally localized atomic visual actions, arXiv:1705.08421, 2017.
  11. W. Kay et al., The kinetics human action video dataset, arXiv:1705.06950, 2017.
  12. J. Moon et al., Extensible hierarchical method of detecting interactive actions for video understanding, ETRI J. 39 (2017), 502-513. https://doi.org/10.4218/etrij.17.0116.0054
  13. L. Cewu, J. Shi, and J. Jia. Abnormal event detection at 150 FPS in MATLAB, in Proc. Int. Conf. Comput. Vision (ICCV), Sydney, Australia, Dec. 1-8, 2013, pp. 2720-2727.
  14. K. Yun, Y. Yoo, and J.Y. Choi, Motion interaction field for detection of abnormal interactions, Mach. Vis. Appl. 28 (2017), 157-171. https://doi.org/10.1007/s00138-016-0816-0
  15. R. Csordas, L. Havasi, and T. Sziranyi, Detecting objects thrown over Fence in outdoor scenes, in Proc. Int. Conf. Comput. Vision Theory Applicat. (VISAPP), Berlin, Germany, 2015, pp. 593-599.
  16. S. Mahankali et al., Identification of illegal garbage dumping with video analytics, in Proc. Int. Conf. Adv. Comput., Commun. Inf. (ICACCI), Bangalore, India, Sept. 19-22, 2018, pp. 2403-2407.
  17. H. Begur et al., An edge‐based smart mobile service system for illegal dumping detection and monitoring in San Jose, in IEEE SmartWorld Ubiquitous Intell. Comput., Adv. Trusted Comput., Scalable Comput. Commun., Cloud Big Data Comput., Internet People Smart City Innovation, SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI, San Francisco, CA, USA, Aug. 4-8, 2017, pp. 1-6.
  18. A. Dabholkar et al., Smart illegal dumping detection, in Proc. IEEE Int. Conf. Big Data Comput. Service Applicat., San Francisco, CA, USA, Apr. 6-9, 2017, pp. 255-260.
  19. E. Esen, M.A. Arabaci, and M. Soysal, Fight detection in surveillance videos, in Proc. Content‐Based Multimedia Indexing (CBMI), Veszprem, Hungary, June 17-19, 2013, pp. 131-135.
  20. Z. Zhang, C. Conly, and V. Athitsos, A survey on vision‐based fall detection, in Proc. ACM Int. Conf. Pervasive Technol. Related Assistive Environ., Corfu, Greece, July 1-3, 2015, pp. 1-7.
  21. X. Zhou, C. Yang, and Y. Weichuan, Moving object detection by detecting contiguous outliers in the low‐rank representation, IEEE Trans. Pattern. Anal. Mach. Intell. 35 (2013), 597-610. https://doi.org/10.1109/TPAMI.2012.132
  22. S. Javed et al., Background‐foreground modeling based on spatiotemporal sparse subspace clustering, IEEE Trans. Image Process. 26 (2017), 5840-5854. https://doi.org/10.1109/TIP.2017.2746268
  23. B. Heo, K. Yun, and J.Y. Choi, Appearance and motion based deep learning architecture for moving object detection in moving camera, in Proc. Int. Conf. Image Process. (ICIP), Beijing, China, Sept. 17-20, 2017, pp. 1827-1831.
  24. T.P. Nguyen et al., Change detection by training a triplet network for motion feature extraction, IEEE Trans. Circuits Syst. Video Technol. 29 (2019), 433-446. https://doi.org/10.1109/TCSVT.2018.2795657
  25. K. Yun, J. Lim, and J.Y. Choi, Scene conditional background update for moving object detection in a moving camera, Pattern Recogn. Lett. 88 (2017), 57-63. https://doi.org/10.1016/j.patrec.2017.01.017
  26. T. Yi Lin et al., Microsoft COCO: Common objects in context, in Proc. Eur. Conf. Comput. Vision (ECCV), Zurich, Switxerland, Sept. 6-12, 2014, pp. 740-755.
  27. M. Andriluka et al., 2D human pose estimation: new benchmark and state of the art analysis, in Proc. Comput. Vision Pattern Recogn. (CVPR), Columbus, OH, USA, June 23-28, 2014, pp. 3686-3693.
  28. Z. Cao et al., Realtime multi‐person 2D pose estimation using part affinity fields, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1302-1310.
  29. G. Rogez, P. Weinzaepfel, and C. Schmid, LCR‐Net: Localization‐classification‐regression for human pose, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1216-1224.
  30. H. Yoo et al., Online scheme for multiple camera multiple target tracking based on multiple hypothesis tracking, IEEE Trans. Circuits Syst. Video Technol. 27 (2017), 454-469. https://doi.org/10.1109/TCSVT.2016.2593619
  31. H.W. Kuhn, The Hungarian method for the assignment problem, Naval Research Logistics Quarterly 2 (1955), 83-97. https://doi.org/10.1002/nav.3800020109
  32. P.F. Felzenszwalb, R.B. Girshick, and D.A. McAllester, Cascade object detection with deformable part models, in Proc. Comput. Vision Pattern Recogn. (CVPR), San Francisco, CA, USA, June 13-18, 2010, pp. 2241-2248.
  33. J. Sklansky, Finding the convex hull of a simple polygon, Pattern Recogn. Lett. 1 (1982), 79-83. https://doi.org/10.1016/0167-8655(82)90016-2
  34. K. Kang et al., Invariant‐feature based object tracking using discrete dynamic swarm optimization, ETRI J. 39 (2017), 151-162. https://doi.org/10.4218/etrij.17.0116.0584
  35. M. E. Yildirim et al., Direction‐based modified particle filter for vehicle tracking, ETRI J. 38 (2016), 356-365. https://doi.org/10.4218/etrij.16.0115.0181
  36. S. Yun et al., Action‐decision networks for visual tracking with deep reinforcement learning, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1349-1358.
  37. J. F. Henriques et al., High‐speed tracking with kernelized correlation filters, IEEE Trans. Pattern Anal. Mach. Intell. 37 (2015), 583-596. https://doi.org/10.1109/TPAMI.2014.2345390
  38. K. M. Yi et al., Detection of moving objects with non‐stationary cameras in 5.8ms: bringing motion detection to your mobile device, in IEEE Conf. Comput. Vision Pattern Recogn. Workshops (CVPRW), Portland, OR, USA, June 23-28, 2013, pp. 27-34.
  39. C. Cortes and V. Vapnik, Support‐vector networks, Mach Learn. 20 (1995), 273-297. https://doi.org/10.1007/BF00994018
  40. S. Yan, Y. Xiong, and D. Lin, Spatial temporal graph convolutional networks for skeleton‐based action recognition, in Proc. AAAI Conf. Artif. Intell., New Orleans, LA, USA, Feb. 2-7, 2018.
  41. C.‐B. Jin et al., Real‐time action detection in video surveillance using sub‐action descriptor with multi‐cnn, arXiv preprint, (2018), arXiv:1710.03383.

Cited by

  1. Three-stream network with context convolution module for human-object interaction detection vol.42, pp.2, 2019, https://doi.org/10.4218/etrij.2019-0230
  2. Learning to Identify Illegal Landfills through Scene Classification in Aerial Images vol.13, pp.22, 2019, https://doi.org/10.3390/rs13224520
  3. CitiusSynapse: A Deep Learning Framework for Embedded Systems vol.11, pp.23, 2019, https://doi.org/10.3390/app112311570
  4. Role of Wireless Aided Technologies in the Solid Waste Management: A Comprehensive Review vol.13, pp.23, 2021, https://doi.org/10.3390/su132313104