Advanced SearchSearch Tips
NUI/NUX of the Virtual Monitor Concept using the Concentration Indicator and the User`s Physical Features
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
NUI/NUX of the Virtual Monitor Concept using the Concentration Indicator and the User`s Physical Features
Jeon, Chang-hyun; Ahn, So-young; Shin, Dong-il; Shin, Dong-kyoo;
  PDF(new window)
As growing interest in Human-Computer Interaction(HCI), research on HCI has been actively conducted. Also with that, research on Natural User Interface/Natural User eXperience(NUI/NUX) that uses user`s gesture and voice has been actively conducted. In case of NUI/NUX, it needs recognition algorithm such as gesture recognition or voice recognition. However these recognition algorithms have weakness because their implementation is complex and a lot of time are needed in training because they have to go through steps including preprocessing, normalization, feature extraction. Recently, Kinect is launched by Microsoft as NUI/NUX development tool which attracts people`s attention, and studies using Kinect has been conducted. The authors of this paper implemented hand-mouse interface with outstanding intuitiveness using the physical features of a user in a previous study. However, there are weaknesses such as unnatural movement of mouse and low accuracy of mouse functions. In this study, we designed and implemented a hand mouse interface which introduce a new concept called `Virtual monitor` extracting user`s physical features through Kinect in real-time. Virtual monitor means virtual space that can be controlled by hand mouse. It is possible that the coordinate on virtual monitor is accurately mapped onto the coordinate on real monitor. Hand-mouse interface based on virtual monitor concept maintains outstanding intuitiveness that is strength of the previous study and enhance accuracy of mouse functions. Further, we increased accuracy of the interface by recognizing user`s unnecessary actions using his concentration indicator from his encephalogram(EEG) data. In order to evaluate intuitiveness and accuracy of the interface, we experimented it for 50 people from 10s to 50s. As the result of intuitiveness experiment, 84% of subjects learned how to use it within 1 minute. Also, as the result of accuracy experiment, accuracy of mouse functions (drag(80.4%), click(80%), double-click(76.7%)) is shown. The intuitiveness and accuracy of the proposed hand-mouse interface is checked through experiment, this is expected to be a good example of the interface for controlling the system by hand in the future.
NUI/NUX;Hand mouse;Virtual Monitor;EEG;Concentration indicator;
 Cited by
W. Bland, T. Naughton, G. Vallee, and S. L. Scott, "Design and implementation of a menu based oscar command line interface," in High Performance Computing Systems and Applications, pp. 25-25, 2007.

M. Park, "A study on the Research on an effective graphic interface design of the web environment for the people with disability-Focused on the people with hearing impairment," Master's Degree, The Department of Design, Sejong University, Seoul, 2005.

M. N. K. Boulos, B. J. Blanchard, C. Walker, J. Montero, A. Tripathy, and R. Gutierrez-Osuna, "Web GIS in practice X: a Microsoft Kinect natural user interface for Google Earth navigation," International journal of health geographics, vol. 10, no. 1, pp. 45, 2011. crossref(new window)

M. F. Shiratuddin and K. W. Wong, "Non-contact multi-hand gestures interaction techniques for architectural design in a virtual environment," in Information Technology and Multimedia (ICIM), pp. 1-6, 2011. crossref(new window)

I. Chiang, J.-C. Tsai, and S.-T. Chen, "Using Xbox 360 kinect games on enhancing visual performance skills on institutionalized older adults with wheelchairs," in Digital Game and Intelligent Toy Enhanced Learning (DIGITEL), pp. 263-267, 2012. crossref(new window)

M. Ahn, and S. Jun, "Brain-Computer Interface System of Principles and technology trends." The Korean Institute of Information Scientists and Engineers, vol. 29, no. 4, pp. 42-53, 2011.

K. Lee, T. Lee, and S. Lee, "Motor Imagery Brain Signal Analysis for EEG-based Mouse Control," Korean Journal of Cognitive Science, vol. 21, no. 2, pp. 309-338, 2010. crossref(new window)

D. J. Sturman and D. Zeltzer, "A survey of glove-based input," Computer Graphics and Applications, IEEE, vol. 14, no. 1, pp. 30-39, 1994. crossref(new window)

F.-S. Chen, C.-M. Fu, and C.-L. Huang, "Hand gesture recognition using a real-time tracking method and hidden Markov models," Image and vision computing, vol. 21, no. 8, pp. 745-758, 2003. crossref(new window)

J. Kim, "Bio-mimetic Recognition of Action Sequence using Unsupervised Learning," Journal of Internet Computing and Service, vol. 15, no. 4, pp. 9-20, 2014. crossref(new window)

J. Kim, "BoF based Action Recognition using Spatio-Temporal 2D Descriptor," Journal of Internet Computing and Service, vol. 16, no. 3, pp. 21-32, 2015. crossref(new window)

T. Baudel and M. Beaudouin-Lafon, "Charade: remote control of objects using free-hand gestures," Communications of the ACM, vol. 36, no. 7, pp. 28-35, 1993. crossref(new window)

A. A. Argyros and M. I. Lourakis, "Vision-based interpretation of hand gestures for remote control of a computer mouse," in Computer Vision in Human-Computer Interaction, ed: Springer, pp. 40-51. 2006. crossref(new window)

G. Lee, D. Shin, and D. Shin, "NUI/NUX framework based on intuitive hand motion," Journal of Internet Computing and Services, vol. 15, no. 3, pp. 11-19, 2014. crossref(new window)

J. O. Lubar and J. F. Lubar, "Electroencephalographic biofeedback of SMR and beta for treatment of attention deficit disorders in a clinical setting," Biofeedback and self-regulation, vol. 9, no. 1, pp. 1-23, 1984. crossref(new window)

M. Sterman, "Sensorimotor EEG operant conditioning: Experimental and clinical effects," The Pavlovian Journal of Biological Science: Official Journal of the Pavlovian, vol. 12, no. 2, pp. 63-92, 1977.

R. Kjeldsen and J. Kender, "Toward the use of gesture in traditional user interfaces," in Automatic Face and Gesture Recognition, pp. 151-156, 1996. crossref(new window)

J. Lee, J. Lee, Y. Myeong, J. Lee, and H. Seong, "Development of Motion Recognition Hand-Mouse Using OpenCV," Telcommunications Research Institue Proceedings, Sangji Uinversity, vol. 7, no. 2, pp. 15-19, 2011.

E. Sanchez-Nielsen, L. Anton-Canalis, and C. Guerra-Artal, "An autonomous and user-independent hand posture recognition system for vision-based interface tasks," in Current Topics in Artificial Intelligence, ed: Springer, pp. 113-122, 2006. crossref(new window)

D. Oh and K. Hong, "Studies on the quantification of relaxation numbers using EEG," HCI 2014, pp. 853-856, 2014.

S. Kang, C. Kim, and W. Son, "Developing User-friendly Hand Mouse Interface via Gesture Recognition," The Korean Society of Broadcast Engineers, pp. 129-132, 2009.