Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time Compressed Video Acquisition System for Stereo 360 VR

Stereo 360 VR을 위한 실시간 압축 영상 획득 시스템

  • Choi, Minsu (Dept. of Image Engineering, Graduate School of Advanced Imaging Science, Multimedia and Film, Chung-Ang University) ;
  • Paik, Joonki (Dept. of Image Engineering, Graduate School of Advanced Imaging Science, Multimedia and Film, Chung-Ang University)
  • 최민수 (중앙대학교 첨단영상대학원 영상학과) ;
  • 백준기 (중앙대학교 첨단영상대학원 영상학과)
  • Received : 2019.09.05
  • Accepted : 2019.11.12
  • Published : 2019.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, Stereo 4K@60fps 360 VR real-time video capture system which consists of video stream capture, video encoding and stitching module is been designed. The system captures stereo 4K@60fps 360 VR video by stitching 6 of 2K@60fps stream which are captured through HDMI interface from 6 cameras in real-time. In video capture phase, video is captured from each camera using multi-thread in real-time. In video encoding phase, raw frame memory transmission and parallel encoding are used to reduce the resource usage in data transmission between video capture and video stitching modules. In video stitching phase, Real-time stitching is secured by stitching calibration preprocessing.

본 논문에서는 Stereo 4K@60fps 360 VR 실시간 영상 획득 시스템을 영상 스트림 획득과 영상 인코딩(encoding), 영상 스티칭(stitching) 모듈로 나누어 설계하였다. 시스템은 6대의 카메라로부터 HDMI Interface를 통해 획득된 6개의 2K@60fps의 영상 스트림을 스티칭을 통하여 실시간으로 Stereo 4K@60fps 360 VR 영상을 획득한다. 영상 획득 단계에서는 멀티 스레드(Multi-Thread)를 이용하여 각 카메라로부터 실시간으로 영상 스트림을 획득하였다. 영상 인코딩 단계에서는 영상 획득과 영상 스티칭 모듈 간의 전송 리소스를 줄이기 위하여 멀티 스레드를 이용한 Raw Frame 메모리 전송과 병렬 인코딩을 하였다. 영상 스티칭 단계에서는 스티칭 Calibration 전처리작업을 통하여 스티칭 실시간성을 확보하였다.

Keywords

References

  1. J. Jang, H. Kim, J. Yang, S. Yoon, W. Park, "Design of 8K 360 VR Video edge streaming system based on IPTV set-top box", The 9th International Conference on Convergence Technology in 2019, pp.216-217, July 2017.
  2. H. Kim, W. Choi, S. Yang, "Implementation of High Quality 360 VR Video Low-latency Live Streaming System using Multi Level Tile Caching based on MPEG DASH SRD", Journal of Korea Multimedia Society, Vol.21, No.8, pp.942-951, Aug 2018. https://doi.org/10.9717/KMMS.2018.21.8.942
  3. H. Kim, "A Study on the 16K 360 VR Tiled Video Streaming based on Edge Computing for Smart Home Media Service", Doctor's Thesis of Kwangwoon University, Seoul, Korea, 2019.
  4. H. Kim, T. Le, E. Ryu, "360-degree video offloading using millimeter-wave communication for cyberphysical system", Transactions of Emerging Telecommunications Technologies, Vol.30, No.4, Apr 2018.
  5. J. Li, M. Fan, G. Wang, X. Li, R, Sun, "Panorama video stitching system based on VR Works 360 video", 2018 Chinese Automation Congress, pp.715-720, Nov 2018.
  6. NVIDIA VRWorks -360 Video, https://developer.nvidia.com/vrworks/vrworks-360video/ (accessed Aug 2019)
  7. NVIDIA Titan V, https://www.nvidia.com/en-us/titan/titan-v/ (accessed Aug 2019)
  8. NVIDIA Titan Xp, https://www.nvidia.com/en-us/titan/titan-xp/ (accessed Aug 2019)
  9. High performance video encoding with NVIDIA GPUs, https://goo.gl/Bdjdgm (accessed Aug 2019)