Journal of Multimedia Information System

한국멀티미디어학회 (Korea Multimedia Society)
권호 표지
DOI QR코드

DOI QR Code

FPGA-based Hardware Prediction Rendering for Low-Latency Touch Platform

  • Song, Seok Bin (Dept. of Computer Engineering, Seo-Kyeong University) ;
  • Kim, Jin Heon (Dept. of Electronic Computer Engineering, Seo-Kyeong University)
  • 투고 : 2018.03.10
  • 심사 : 2018.03.19
  • 발행 : 2018.03.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

The delay between input action and visual interface feedback ("Latency") in a touchscreen inking task reduces the user's performance. When the latency is less than 2.38ms, the user cannot perceive the latency in dragging task. This value is difficult to achieve on recent touchscreens and general purpose computers. So, methods of predicting touch points to reduce perceptible latency has been proposed. In general, touch points prediction is not perfect. When using point prediction, feedback of the predicted points is displayed on the screen, after a while, erased when the actual points are displayed. When this task is implemented by software, it causes additional latency to work to erase predicted points feedback. It therefore propose a platform for rendering point prediction feedback without additional latency by the FPGA. This platform transmits input points and HDMI signals rendering feedback of input points to the FPGA. The FPGA draws the feedback of points predicted based on the input points on the HDMI and displays the screen. Since hardware rendering changes the HDMI signal every frame, it does not require erasing work and rendering can be done within an early time regardless of the amount of rendering, so we will reduce the latency.

키워드

E1MTCD_2018_v5n1_59_f0001.png 이미지

Fig. 1. System configuration diagram.

E1MTCD_2018_v5n1_59_f0002.png 이미지

Fig. 2. Software rendering.

E1MTCD_2018_v5n1_59_f0003.png 이미지

Fig. 3. Software rendering and hardware prediction rendering.

E1MTCD_2018_v5n1_59_f0004.png 이미지

Fig. 4. Touch point predict algorithm.

Table. 1. Latency differences between hardware prediction line rendering and software prediction line rendering.

E1MTCD_2018_v5n1_59_t0001.png 이미지

참고문헌

  1. Meehan, M., Razzaque, S., Whitton, M. C., and Brooks, F. P., "Effect of Latency on Presence in Stressful Virtual Environments," IEEE VR, pp. 138-141, March 2003.
  2. K. Kin, M. Agrawala, and T. DeRose, "Determining the Benefits of Directr-touch, bimanual, and Multifinger Input on a Multitouch Workstation." in Proceeding of Graphics Interface, pp. 119-127, May. 2009.
  3. R. Jota, A. Ng, P. Dietz, and D. Wig- dor, "How fast is fast enough?: User Perception of Latency & Latency Improvements in direct and Indirect Touch," in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing System, pp 1827-1836, April. 2015.
  4. Ng, A., Lepinski, J., Wigdor, D., Sanders, S., & Dietz, P., "Designing for low-latency direct-touch input." in Proceedings of the 25th annual ACM symposium on User interface software and technology, pp. 453-464, October. 2012.
  5. Jota, R., Ng, A., Dietz, P., & Wigdor, D., "How fast is fast enough? : a study of the effects of latency in direct-touch pointing tasks", in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2291-2300, April. 2013.
  6. Takeshi Asano, Ehud Sharlin, Yoshifumi Kitamura, Kazuki Takashima, and Fumio Kishino. "Predictive interaction using the delphian desktop," in Proceedings of the 18th Annual ACM Symposium on User Interface Soft- ware and Technology, pp. 133-141, October. 2005.
  7. Jun Han Yoon and Jin Heon Kim, "An Implementation of High Speed Rendering to Process Touch Screen Multiple Inputs Based on FPGA," in procedding of the Journal of Korea Multimedia Society 20(11), pp. 1803-1810, November 2017.
  8. J. S Park, J. M. Lim, and K. W. Kyeong, "Tangible Touch Interface Technology Trend," Korean Information Processing Society Review, Vol. 20, No 1, pp. 45-53, 2013
  9. Kim, B., and Lim, Y. "Mobile terminal and touch coordinate predicting method thereof," WO Patent App, Aug. 2014.
  10. LINCOLN, J. "Position lag reduction for computer drawing," US Patent App, Oct. 2013
  11. Mathieu Nancel, Daniel Vogel, Bruno De Araujo, Ricardo Jota, and Gery Casiez, "Next-Point Prediction Metrics for Perceived Spatial Errors," in Proceedings of the 29th Annual Symposium on User Interface Software and Technology, 2016.
  12. Kaaresoja, Topi Johannes, "Latency guidelines for touchscreen virtual button feedback," School of Computing Science College of Science and Engineering, University of Glasgow, PhD thesis, March 2015
  13. Joseph J. LaViola., "Double Exponential Smoothing: An Alternative to Kalman Filter-based Predictive Tracking," In Proceedings of the Workshop on Virtual Environments 2003 (EGVE'03), pp. 199-206. 2003
  14. David Asselborn, and Jan Borchers, "A Predictive Approach to Compensate Latency of Tangibles on Capacitive Multi-Touch-Displays," Media Computing Group, Computer Science Department PWTH Aachen University, July 2016
  15. J.W. Park, J.Y. Ko, J.H. Park, M.H. Hong, Y.H. Lee and J.C. Shim, "A Wireless Temperature Control System based on FPGA," Journal of Korea Multimedia Society, Vol. 15, No. 7, pp. 920-930, 2012 https://doi.org/10.9717/kmms.2012.15.7.920
  16. Xilinx, http://www.xilinx.com/products/silicon-devices/fpga/artix-7.html#documentation, (accessed Jul. 8. 2018).
  17. Digilent, https://store.digilentinc.com/nexys-video-artix-7-fpga-trainer-board-for-multimedia-applications/ (accessed March. 8. 2018)
  18. Raspberry Pi Foundation, https://www.raspberrypi.org/products/ (accessed March. 8. 2018)
  19. AFO, http://afoi.co.kr/business/module.php, (accessed March. 8. 2018)
  20. Qt, http://wiki.qt.io/Main (accessed March. 8. 2018)
  21. Samsung, https://www.samsung.com/us/business/products/displays/standalone/ed-series/ed-e-series-65-lh65edeplgc-go/?CID=AFL-hq-mul-0813-11000758 (accessed March. 8. 2018)
  22. Peter Tsoi and Jacob Xiao. "Advanced touch input on iOS," Technical report, Apple Inc., 2015.
  23. IPhone X, https://www.apple.com/kr/iphone-x/specs/ (accessed March. 8. 2018)