DOI QR코드

DOI QR Code

Development of Performance Evaluation Formula for Deep Learning Image Analysis System

딥러닝 영상분석 시스템의 성능평가 산정식 개발

  • Hyun Ho Son (Dept. of Traffic Safety & Facility, Korea Road Traffic Authority) ;
  • Yun Sang Kim (Pyeongtack police station) ;
  • Choul Ki Lee (Dept of Transportation Eng, Ajou University)
  • 손현호 (도로교통공단 경기지부) ;
  • 김윤상 (평택경찰서 교통과) ;
  • 이철기 (아주대학교 교통시스템공학과)
  • Received : 2023.06.16
  • Accepted : 2023.07.06
  • Published : 2023.08.31

Abstract

Urban traffic information is collected by various systems such as VDS, DSRC, and radar. Recently, with the development of deep learning technology, smart intersection systems are expanding, are more widely distributed, and it is possible to collect a variety of information such as traffic volume, and vehicle type and speed. However, as a result of reviewing related literature, the performance evaluation criteria so far are rbs-based evaluation systems that do not consider the deep learning area, and only consider the percent error of 'reference value-measured value'. Therefore, a new performance evaluation method is needed. Therefore, in this study, individual error, interval error, and overall error are calculated by using a formula that considers deep learning performance indicators such as precision and recall based on data ratio and weight. As a result, error rates for measurement value 1 were 3.99 and 3.54, and rates for measurement value 2 were 5.34 and 5.07.

도시부 교통정보 수집은 VDS, DSRC, 레이더 등 다양한 시스템에 의해 수집되고 있다. 최근 딥러닝 기술의 발전으로 스마트교차로시스템이 확대 보급되고 있으며 교통량, 속도, 차종 등 다양한 정보수집이 가능하다. 그러나 관련 문헌을 고찰한 결과 지금까지의 성능평가 기준은 딥러닝 영역을 고려하지 않은 RBS기반 평가체계로 '기준값-측정값'의 퍼센트 오차만 고려하고 있어 기존 평가방식으로는 딥러닝 부분의 평가를 수행할 수 없어 새로운 성능평가 방법이 필요하다. 따라서, 본 연구에서는 데이터 비율 및 가중치를 고려하여 Precision과 Recall 등 딥러닝 성능지표를 고려한 오차산정식을 개발하여 개별오차와 구간 오차, 전체오차를 산정하였다. 연구결과, 측정값 1의 오차율은 3.99와 3.54, 측정값 2는 5.34와 5.07로 기존 산정식과 오차율에 차이가 있는 것으로 나타났으며, 반복측정 분석결과 개발 산정식이 우수한 것으로 나타났다.

Keywords

References

  1. AI times, https://www.aitimes.com, 2023.06.14.
  2. Colin, R. B.(1972), "Is Pitman Closeness a Reasonable Criterion Comment", Journal of the American Statistical Association, vol. 88, no. 421, pp.72-74.
  3. Curran, P. J., Stephen, G. W. and John, F. F.(1996), "The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis", American Psychological Association Psychological Methods, vol. 1, no. 1, pp.16-29. https://doi.org/10.1037/1082-989X.1.1.16
  4. Dreamsherpa, https://m.blog.naver.com, 2023.05.24.
  5. Jang, J. H., Baek, N. C. and Kang, W. U.(2004), "Study on Vehicle Performance Test Method", The Korea Institute of Intelligent Transport Systems Conference, p.199.
  6. Kim, S. H., Jo, Y. J., Joung, Y. T. and Sin, K. W.(2018), "Deep learning-based smart intersection system evaluation", The 2018 Korean Institute of ITS Conference, p.589.
  7. Kline, R. B.(2015), Principles and practice of structural equation modeling, Guilford Publications, NewYork, USA, p.534.
  8. Korea Institute of Civil Engineering and Building Technology(2006), Research on practical use of ITS equipment/system performance evaluation and inspection/calibration system, p.32.
  9. Kwon, C. H.(2004), "A Study On Camera-Detector System for Traffic Control in Korea", The Society of Digital Policy & Management 2004 Spring Conference, pp.593-608.
  10. Lee, H. P.(2012), "Development of an Performance Evaluation Method for Vehicle Detector Speed Measurement Applying Uncertainty in Measurement", International Journal of Highway Engineering, vol. 14, no. 2, pp.165-174. https://doi.org/10.7855/IJHE.2012.14.2.165
  11. Oh, J. S.(2021), "A Study on the Performance Characteristics of Image Vehicle Detectors Depending on the Environment", The Korea Contents Association, vol. 21, no. 1, pp.123-126.
  12. Park, H. L., Byun, S. H. and Lee, H. S.(2020), "Application of Deep Learning Method for Real-Time Traffic Analysis using UAV", Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, vol. 38, no. 4, pp.356-360.
  13. Park, Y. K.(2020), "Traffic Data Calculation Solution for Moving Vehicles using Vision Tracking", The Journal of The Korea Institute of Intelligent Transport Systems, vol. 19, no. 5, pp.99-103. https://doi.org/10.12815/kits.2020.19.5.97
  14. Simson, E. H.(1951), "The Interpretation of Interaction in Contingency Tables", Journal of the Royal Statistical Society Series B, vol. 13, no. 2, pp.238-241.
  15. Snedecor, G. W. and Cochran, W. G.(1980), "Statistical Methods (7th ed.)", Iowa State University Press, Ames, vol. 7, p.507.
  16. Sorower, M. S.(2010), "A literature survey on algorithms for multi-label learning", Oregon State University Corvallis, vol. 18, no. 1, pp.3-8.
  17. Yaw, O. A.(2017), "Automated Vehicle Recognition with Deep Convolutional Neural Networks", Information Technology, Geospatial Information, and Advanced Computing, vol. 2645, no. 1, pp.113-122.