Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Online Multi-Object Tracking by Learning Discriminative Appearance with Fourier Transform and Partial Least Square Analysis

  • Lee, Seong-Ho (Dept. of Computer Science and Engineering, Incheon National University) ;
  • Bae, Seung-Hwan (Dept. of Computer Science and Engineering, Incheon National University)
  • Received : 2020.01.14
  • Accepted : 2020.02.03
  • Published : 2020.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we solve an online multi-object problem which finds object states (i.e. locations and sizes) while conserving their identifications in online-provided images and detections. We handle this problem based on a tracking-by-detection approach by linking (or associating) detections between frames. For more accurate online association, we propose novel online appearance learning with discrete fourier transform and partial least square analysis (PLS). We first transform each object image into a Fourier image in order to extract meaningful features on a frequency domain. We then learn PLS subspaces which can discriminate frequency features of different objects. In addition, we incorporate the proposed appearance learning into the recent confidence-based association method, and extensively compare our methods with the state-of-the-art methods on MOT benchmark challenge datasets.

본 연구는 온라인 다중 객체 추적 환경에서 모든 객체의 상태(예. 위치 및 크기) 및 identifications (IDs)를 추적하는 문제를 다룬다. 프레임들 간 검출 결과들을 연관하여 객체들의 궤도를 점진적으로 완성하는 tracking-by-detection 접근법을 기반으로 온라인 다중 객체 추적 문제를 해결하고자 한다. 정확한 온라인 연관을 수행하기 위해 이산 푸리에 변환과 부분 최소 제곱법(partial least square, PLS) 분석을 기반으로 하는 새로운 온라인 외형 학습 방법을 제안한다. 즉, 먼저 주파수 도메인에서 추적에 용이한 객체 특징량을 추출하기 위해 추적 객체에 대한 이미지를 푸리에 이미지로 변환한다. 나아가 객체간의 주파수 특징을 보다 잘 구별할 수 있도록 PLS기반 부분 공간을 학습한다. 제안된 외형 학습을 최신 신뢰도 기반 연관 기법과 결합하였고, 다중 객체 추적평가 분야에서 국제적으로 공인된 MOT 벤치마크 챌린지 데이터 셋에서 최신 다중 객체 추적 알고리즘과 비교평가를 수행하였다.

Keywords

Acknowledgement

Supported by : Incheon National University

This work was supported by Incheon National University (International Cooperative) Research Grant in 2019

References

  1. D. Xie, W. Hu, T. Tan, and J. Peng, "A multi-object tracking system for surveillance video analysis," Proceedings of the 17th International Conference on Pattern Recognition, Vol. 4., pp. 767-770, Aug. 2004, DOI: 10.1109/ICPR.2004.1333885
  2. Y. Ye, L. Fu and B. Li, "Object detection and tracking using multi-layer laser for autonomous urban driving," IEEE International Conference on Intelligent Transportation Systems, pp. 259-264. Nov. 2016, DOI: 10.1109/ITSC.2016.7795564
  3. K. Fragkiadaki and J. Shi, "Detection free tracking: Exploiting motion and topology for segmenting and tracking under entanglement,", IEEE Conference on Computer Vision and Pattern Recognition, pp. 2073-2080, June 2011, DOI: 10.1109/CVPR.2011.5995366
  4. S. Bae and K. Yoon, "Confidence-Based Data Association and Discriminative Deep Appearance Learning for Robust Online Multi-Object Tracking,", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol 40, no. 3, pp. 595-610, Nov. 2018, DOI: 10.1109/TPAMI.2017.2691769
  5. K. He, G. Gkioxari, P. Dollar, and R. Girshick "Mask R-CNN", Proceedings of the IEEE international conference on computer vision, pp. 2961-2969 , Oct., 2017. DOI: 10.1109/iccv.2017.322
  6. D. Bolya, C. Zhou, F. Xiao, and J. Lee "YOLACT: Real-time Instance Segmentation", Proceedings of the IEEE international conference on computer vision, pp. 9157-9166 , Oct., 2019. arXiv:1904.02689
  7. C. Kim, F. Li, A. Ciptadi, and J. M.. Rehg "Multiple hypothesis tracking revisited", Proceedings of the IEEE international conference on computer vision, pp. 4696-4704, Dec. 2015, DOI: 10.1109/iccv.2015.533
  8. P. Bergmann, T. Meinhardt, and L. Leal-Taixe, "Tracking without bells and whistles", Proceedings of the IEEE international conference on computer vision, pp. 941-951 , Oct. 2019. arXiv:1903.05625
  9. X. Li, K. Wang, W. Wang and Y. Li, "A multiple object tracking method using Kalman filter," The 2010 IEEE International Conference on Information and Automation, Harbin, pp. 1862-1866., June 2010, DOI: 10.1109/ICINFA.2010.5512258
  10. J. Yoon, M.. Yang, J. Lim, and K. Yoon, "Bayesian Multi-Object Tracking Using Motion Context from Multiple Objects", The 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 33-40, Jan. 2015, DOI: 10.1109/WACV.2015.12
  11. B. Benfold, and I. Reid, "Stable multi-target tracking in real-time surveillance video", IEEE Conference on Computer Vision and Pattern Recognition, pp. 3457-3464, June 2011, DOI: 10.1109/CVPR.2011.5995667
  12. J. Ge, Y. Luo, and G. Tei, "Real-time pedestrian detection and tracking at nighttime for driver-assistance systems" IEEE Trans. on Intelligent Transportation Systems, vol. 10, no. 2, pp. 283-298, June 2009, DOI: 10.1109/TITS.2009.2018961
  13. W. Hu, X. Li, W. Luo, X. Zhang, S. Maybank and Z. Zhang, "Single and Multiple Object Tracking Using Log-Euclidean Riemannian Subspace and Block-Division Appearance Model," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 34, no. 12, pp. 2420-2440, Dec. 2012. DOI: 10.1109/TPAMI.2012.42
  14. S. Lee, M. Kim and S. Bae, "Learning Discriminative Appearance Models for Online Multi-Object Tracking With Appearance Discriminability Measures," IEEE Access, vol. 6, pp. 67316-67328, Nov. 2018. DOI: 10.1109/ACCESS.2018.2879535
  15. J. Son, M. Baek, M. Cho and B. Han, "Multi-object Tracking with Quadruplet Convolutional Neural Networks," IEEE Conference on Computer Vision and Pattern Recognition, pp. 3786-3795, July 2017. DOI: 10.1109/CVPR.2017.403
  16. J. F. Henriques, R. Caseiro, P. Martins and J. Batista, "High-Speed Tracking with Kernelized Correlation Filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 583-596, Aug. 2014. DOI: 10.1109/TPAMI.2014.2345390
  17. Z. He, Z. Zhang and C. Jung, "Fast Fourier Transform Networks for Object Tracking Based on Correlation Filter," IEEE Access, vol. 6, pp. 6594-6601, Jan. 2018. DOI: 10.1109/ACCESS.2018.2790942
  18. F. Bourgeois and J.. C. Lassalle, "An extension of the munkres algorithm for the assignment problem to rectangular matrices," Communications of the ACM, vol. 14, no. 12, pp. 802-804, Dec. 1971. DOI: 10.1145/362919.362945
  19. H. Wold "Path models with latent variables: The NIPALS approach." Quantitative sociology. Academic Press, pp. 307-357, 1975. DOI: 10.1016/b978-0-12-103950-9.50017-4
  20. A. Milan, L. Leal-Taixe, I. D. Reid, S. Roth, and K. Schindler, "MOT16: A benchmark for multi-object tracking," CoRR, Mar. 2016. arXiv:1603.00831
  21. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sep. 2010. DOI: 10.1109/TPAMI.2009.167
  22. N. L. Baisa, "Online multi-object visual tracking using a GM-PHD filter with deep appearance learning," International Conference on Information Fusion, July 2019, arXiv:1907.13347
  23. R. Sanchez-Matilla, F. Poiesi, and A. Cavallaro, "Online multi-target tracking with strong and weak detections," European Conference on Computer Vision, pp. 84-99, Nov. 2016. DOI: 10.1007/978-3-319-48881-3_7
  24. Y. Ban, S. Ba, X. Alameda-Pineda, and R. Horaud, "Tracking Multiple Persons Based on a Variational Bayesian Model," European Conference on Computer Vision , ser. Lecture Notes in Computer Science, vol. 9914, pp. 52-67, Oct. 2016. DOI: 10.1007/978-3-319-48881-3_5
  25. A. Boragule and M. Jeon, "Joint cost minimization for multi-object tracking," IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 1-6, Aug. 2017. DOI: 10.1109/avss.2017.8078481
  26. Y. Song and M. Jeon, "Online multiple object tracking with the hierarchically adopted gm-phd filter using motion and appearance," IEEE Conference on Consumer Electronics-Asia, pp. 1-4. Oct. 2016. DOI: 10.1109/icce-asia.2016.7804800
  27. J. Son, M. Baek, M. Cho, and B. Han, "Multi-object tracking with quadruplet convolutional neural networks," in 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3786-3795, July 2017. DOI: 10.1109/cvpr.2017.403
  28. A. Dehghan, S. M. Assari, and M. Shah, "GMMCP tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking," IEEE Conference on Computer Vision and Pattern Recognition, pp. 4091-4099, June 2015. DOI: 10.1109/cvpr.2015.7299036
  29. H. Pirsiavash, D. Ramanan, and C. C. Fowlkes, "Globally-optimal greedy algorithms for tracking a variable number of objects," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1201-1208, June 2011. DOI: 10.1109/cvpr.2011.5995604