Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Vanishing Line based Lane Detection for Augmented Reality-aided Driver Induction

  • Yun, Jeong-Rok (Spetial Optical Information Research Center, Korea Photonics Technology Institute) ;
  • Lee, Dong-Kil (Spetial Optical Information Research Center, Korea Photonics Technology Institute) ;
  • Chun, Sung-Kuk (Spetial Optical Information Research Center, Korea Photonics Technology Institute) ;
  • Hong, Sung-Hoon (School of Electronic and Computer Science, Chonnam National University)
  • Received : 2018.12.18
  • Accepted : 2019.01.19
  • Published : 2019.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose the augmented reality(AR) based driving navigation based on robust lane detection method to dynamic environment changes. The proposed technique uses the detected lane position as a marker which is a key element for enhancing driving information. We propose Symmetrical Local Threshold(SLT) algorithm which is able to robustly detect lane to dynamic illumination environment change such as shadows. In addition, by using Morphology operation and Connected Component Analysis(CCA) algorithm, it is possible to minimize noises in the image, and Region Of Interest(ROI) is defined through region division using a straight line passing through several vanishing points We also propose the augmented reality aided visualization method for Interchange(IC) and driving navigation using reference point detection based on the detected lane coordinates inside and outside the ROI. Validation experiments were carried out to assess the accuracy and robustness of the proposed system in vairous environment changes. The average accuracy of the proposed system in daytime, nighttime, rainy day, and cloudy day is 79.3% on 4600 images. The results of the proposed system for AR based IC and driving navigation were also presented. We are hopeful that the proposed research will open a new discussion on AR based driving navigation platforms, and thus, that such efforts will enrich the autonomous vehicle services in the near future.

Keywords

CPTSCQ_2019_v24n1_73_f0001.png 이미지

Fig. 1. System flowchart

CPTSCQ_2019_v24n1_73_f0002.png 이미지

Fig. 2. Image conversion process

CPTSCQ_2019_v24n1_73_f0003.png 이미지

Fig. 3. Examples of erosion operations

CPTSCQ_2019_v24n1_73_f0004.png 이미지

Fig. 4. Examples of dilatation operations

CPTSCQ_2019_v24n1_73_f0005.png 이미지

Fig. 5. Morphology result

CPTSCQ_2019_v24n1_73_f0006.png 이미지

Fig. 6. Example of preprocessing results based on Connected Component Analysis

CPTSCQ_2019_v24n1_73_f0007.png 이미지

Fig. 7. Binary image region dividing results

CPTSCQ_2019_v24n1_73_f0008.png 이미지

Fig. 8. Example of Hough Transform-based Line Detection Results

CPTSCQ_2019_v24n1_73_f0009.png 이미지

Fig. 9. Mean coordinates of vanishing points and vanishing points

CPTSCQ_2019_v24n1_73_f0010.png 이미지

Fig. 10. ROI area

CPTSCQ_2019_v24n1_73_f0011.png 이미지

Fig. 11. Example of proposed lane detection method result

CPTSCQ_2019_v24n1_73_f0012.png 이미지

Fig. 12. Lane detection test result

CPTSCQ_2019_v24n1_73_f0013.png 이미지

Fig. 13. Lane detection failure result

CPTSCQ_2019_v24n1_73_f0014.png 이미지

Fig. 14. HUD Navigation

CPTSCQ_2019_v24n1_73_f0015.png 이미지

Fig. 15. ROI area and IC lane

CPTSCQ_2019_v24n1_73_f0016.png 이미지

Fig. 16. IC lane detection result

CPTSCQ_2019_v24n1_73_f0017.png 이미지

Fig. 17. IC lane detection failure image

CPTSCQ_2019_v24n1_73_f0018.png 이미지

Fig. 18. ROI area and ROI Inner lane

CPTSCQ_2019_v24n1_73_f0019.png 이미지

Fig. 19. Example of visualization of Augmented Reality

CPTSCQ_2019_v24n1_73_f0020.png 이미지

Fig. 20. Implement augmented reality

CPTSCQ_2019_v24n1_73_f0021.png 이미지

Fig. 21 Driving navigation visualization

CPTSCQ_2019_v24n1_73_f0022.png 이미지

Fig. 22. Driving navigation visualization on image with large illumination change

CPTSCQ_2019_v24n1_73_f0023.png 이미지

Fig. 23. Driving navigation visualization on IC images

CPTSCQ_2019_v24n1_73_f0024.png 이미지

Fig. 24. Lane detection failure image visualization

References

  1. H. Seunghoi, B. Sunwo, and K. Jungha "Lane Keping System using lane detection and Pure Pursuit algorithm in Automotive road environment," The Korean Society Of Automotive Engineers. pp.716-724, Jun. 2018.
  2. L. Hyeongmin, and L. Sangyoun, "Target Region Detection Using Line Pruning Filter-Based Hough Transform for Augmented Reality," Journal of The Institute of Electronics and Information Engineers, Vol. 54, No. 10, pp.79-87, Oct. 2017. https://doi.org/10.5573/ieie.2017.54.10.79
  3. A. Lee, J. Lee, S. Lee, and J. Choi, "Augmented Reality System using Planar Natural Feature Detection and Its Tracking," Journal of the Institute of Electronics Engineers of Korea SP, vol. 48, no.4, pp.49-58, Jul. 2011.
  4. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, "Recent advances in augmented reality" IEEE computer graphics and applications, Vol. 21, No.6, pp.34-47, 2001. https://doi.org/10.1109/38.963459
  5. Y. Shin, J. Park, S. Shin, G. Lim, S. Song, C. Lee, and J. Chung, "Improvement of Image Classification in Augmented Reality Based on Deep Learning," 2016 Summer Conference of the Institute of Electronics Engineers of Korea, pp.1771-1773, Jun. 2016.
  6. O. Richard, and E. Peter, "Use of the Hough transformation to detect lines and curves in pictures," Communications of the ACM, Vol.15, No.5, pp.11-15, Oct. 1972. https://doi.org/10.1145/361237.361242
  7. K. Taedong, Y. Kang and J. Kyeonghoon, "Robust Lane Detection Algorithm in Shadow Area by using Local Feature Point," The Korean Institute of Broadcast and Media Engineers, pp.194-197, Oct. 2016.
  8. J. Shin, L. Eunryung, K. Keekoo, and L. SooIn, "Lane detection algorithm based on top-view image using random sample consensus algorithm and curve road model," 2014 Sixth International Conference on Ubiquitous and Future Networks, Vol.10, No.1109, pp.8-11, Jul. 2014.
  9. A. Martin, and C. Robert, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, Vol. 24, No.6, pp.381-395, Jun. 1981. https://doi.org/10.1145/358669.358692
  10. S, Minseok, K. Sungho, L. Eunryung, M. Afrakhieh, and P. Miryoung, "A study of Augment Reality (AR) technology for recognition improvement of Vehicle HUD," The Korean Society of Automotive Engineers, pp.1302-1303, Nov. 2016.
  11. K. Beayoung, J. Sungho, B. Chulho, P. Junghoon, and S. Myungwon, "A Study on Workload of Using Telematics while Driving," KSAE, Vol.17, No.2, pp.26-33, Mar. 2009.
  12. O. Youngdal, K. EunJeong, and P. SunHong, "A Study on the Effectiveness of Head Up Display (HUD)," The Korean Society of Automotive Engineers, pp.655-655, May. 2014.
  13. P. Daehyuck, S. Jeonggoo, K. Jihyung, J. Seogsig, L. Junghun, Y. Tae-sup, L. Hye, X. Bin, and L. Younghwan, "A Study on Environmentally Adaptive Real-Time Lane Recognition Using Car Black Box Video Images," KSCI, pp. 187-190, Jul. 2018.
  14. Y. Wang, E. Teoh, and D. Shen, "Lane detection and tracking using B-Snake," Image Vision Comput," Vol.22, No.4, pp.269-280, Apr. 2004. https://doi.org/10.1016/j.imavis.2003.10.003
  15. M. Aly, "Real time Detection of Lane Markers in Urban Streets," IEEE Symposium on Intelligent Vehicles, pp.7-12, June. 2008.
  16. H. Cheng, B. Jeng, P. Tseng, and K. Fan, "Lane Detection With Moving Vehicles in the Traffic Scenes," Intell. Transport. syst, Vol.7, no.4, pp.571-582, Dec. 2006. https://doi.org/10.1109/TITS.2006.883940
  17. A. Abdulhakam, O. Khalifa, and H. Khan, "Real Time Lane Detection for Autonomous Vehicles," International Conference on Computer and Communication Engineering, Vol.10, No.1109, pp. 13-15, Jul. 2008.
  18. Chanho Lee, and Dajun Ding, "An Adaptive Road ROI Determination Algorithm for Lane Detection," IEIE, pp.116-125, Jan. 2014.
  19. X. Zhe, and L. Zhifeng, "A robust lane detection method in the different scenarios," International Conference on Mechatronics and Automation, pp.1358-1363, Aug. 2012.
  20. W. Chunche, H. Shihshinh, and F. Lichen, "Driver assistance system for lane detection and vehicle recognition with night vision," IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3530-3535, Aug. 2005.
  21. B. Benligiray, C. Topal, and C. Akinlar, "Video-Based Lane Detection Using a Fast Vanishing Point Estimation Method," IEEE International Symposium on Multimedia, pp.348-351, Dec. 2012.
  22. H. Kong, J. Audibert, and J. Ponce, "Vanishing point detection for road detection," IEEE International Conference on Computer Vision and Pattern Recognition, pp.96-103, Jun. 2009.
  23. L. Mingfa, L. Yuanyuan, and J. Min "Lane Detection Based on Connection of Various Feature Extraction Methods," Hindawi, pp.13, Jul. 2018.
  24. Y. Hu, J. Liu, S. Gao, and W. Chi "Lane Detection System for Night Scenes" Intelligent and Connected Vehicles Symposium, pp.9, Jan. 2018.
  25. K. Byeoungsu, and K. Whoiyul "Robust Lane Detection Method in Varying Road Conditions" The Institute of Electronics Engeineers Of Korea, pp.88-93, Jan. 2012
  26. Y. Cheng, C. Tseng, C. Fan, N. Hwang, and S. Jeng, "Hierarchical lane detection for different type of roads," IEEE Transactions on Acoustics, Speech and Signal Processing, pp.1349-1352, Mar. 2008.
  27. K. Kluge, S. Lakshmanan, "A deformable template approach to lane detection," IEEE Intelligent Vehicle, pp.54-59, Sep. 1995.
  28. H. Lim, M. Ang, P. Seng, and W. Chin, "Lane-vehicle detection and tracking," Proc. International Multiconference of Engineers and Computer Scientists, pp.18-20, Mar. 2009.
  29. C. Rasmussen, "Grouping dominant orientations for ill-structured road following," Proc. IEEE Comput. Soc. Conf, Comput. Vis. and Pattern Recognition, pp.394-398, Jul. 2004.
  30. C. Lipski, B. Scholz, K. Berger, C. Linz, T. Stich, and M. Magnor, "A Fast and Robust Approach to Lane Marking Detection and Lane Tracking," Proc. IEEE Southwest Symposium on Image Analysis and Interpretation, pp.57-60, Apr. 2008.
  31. T. Veit, J. Tarel, P. Nicolle, and P. Charbonnier, "Evaluation of Road Marking Feature Extraction," 11th International IEEE Conference on Intelligent Transportation Systems, pp.174-181, Dec. 2008.
  32. Y. Bumjae "Virtual Reality, Augmented Reality, and Mixed Reality Overview" Korea Robotics Soclety, pp.3-7, Oct. 2018.
  33. K. Sangjoon, B. Yoonmin, and C. Yoonjoo, "Design and Implementatin of Real-time Augmented Reality Building Information System Combined with 3D Map," Korea Computer Graphics Soclety, pp.39-54, Sep. 2018. https://doi.org/10.15701/kcgs.2018.24.4.39
  34. Y. Youngmi, and J. Seonghwan"Design Plan for Digital Textbooks Applying Augmented Reality Image Recognition Technology, The Korea Contents Soclety, pp.353-363, Jun. 2018.