Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

Classification of Mental States Based on Spatiospectral Patterns of Brain Electrical Activity

  • Hwang, Han-Jeong (Department of Biomedical Engineering, Hanyang University) ;
  • Lim, Jeong-Hwan (Department of Biomedical Engineering, Hanyang University) ;
  • Im, Chang-Hwan (Department of Biomedical Engineering, Hanyang University)
  • Received : 2011.11.29
  • Accepted : 2012.01.28
  • Published : 2012.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Classification of human thought is an emerging research field that may allow us to understand human brain functions and further develop advanced brain-computer interface (BCI) systems. In the present study, we introduce a new approach to classify various mental states from noninvasive electrophysiological recordings of human brain activity. We utilized the full spatial and spectral information contained in the electroencephalography (EEG) signals recorded while a subject is performing a specific mental task. For this, the EEG data were converted into a 2D spatiospectral pattern map, of which each element was filled with 1, 0, and -1 reflecting the degrees of event-related synchronization (ERS) and event-related desynchronization (ERD). We evaluated the similarity between a current (input) 2D pattern map and the template pattern maps (database), by taking the inner-product of pattern matrices. Then, the current 2D pattern map was assigned to a class that demonstrated the highest similarity value. For the verification of our approach, eight participants took part in the present study; their EEG data were recorded while they performed four different cognitive imagery tasks. Consistent ERS/ERD patterns were observed more frequently between trials in the same class than those in different classes, indicating that these spatiospectral pattern maps could be used to classify different mental states. The classification accuracy was evaluated for each participant from both the proposed approach and a conventional mental state classification method based on the inter-hemispheric spectral power asymmetry, using the leave-one-out cross-validation (LOOCV). An average accuracy of 68.13% (${\pm}9.64%$) was attained for the proposed method; whereas an average accuracy of 57% (${\pm}5.68%$) was attained for the conventional method (significance was assessed by the one-tail paired $t$-test, $p$ < 0.01), showing that the proposed simple classification approach might be one of the promising methods in discriminating various mental states.

Keywords

References

  1. V. Abootalebi, M. H. Moradi, and M. A. Khalilzadeh, "A new approach for EEG feature extraction in P300-based lie detection," Comput. Meth. Programs Biomed., vol. 94, pp. 48-57, 2009. https://doi.org/10.1016/j.cmpb.2008.10.001
  2. B. Z. Allison, C. Brunner, V. Kaiser, G. R. Muller-Putz, C. Neuper, and G. Pfurtscheller, "Toward a hybrid brain-computer interface based on imagined movement and visual attention," J. Neural Eng., vol. 7(2), 2010.
  3. H. Bakardjian, T. Tanaka, and A. Cichocki, "Optimization of SSVEP brain responses with application to eight-command Brain-Computer Interface," Neurosci. Lett., vol. 469, pp. 34- 38, 2010. https://doi.org/10.1016/j.neulet.2009.11.039
  4. M. R. Coleman, J. M. Rodd, M. H. Davis, I. S. Johnsrude, D. K. Menon, J. D. Pickard, and A. M. Owen, "Do vegetative patients retain aspects of language comprehension? Evidence from fMRI," Brain, vol. 130, pp. 2494-2507, 2007. https://doi.org/10.1093/brain/awm170
  5. C. Davatzikos, K. Ruparel, Y. Fan, D. G. Shen, M. Acharyya, J. W. Loughead, R. C. Gur, and D. D. Langleben, "Classifying spatial patterns of brain activity with machine learning methods: application to lie detection," Neuroimage, vol. 28, pp. 663-8, 2005. https://doi.org/10.1016/j.neuroimage.2005.08.009
  6. J. Fruitet, D. J. McFarland, and J. R. Wolpaw, "A comparison of regression techniques for a two-dimensional sensorimotor rhythm-based brain-computer interface," J. Neural Eng., vol. 7, 2010.
  7. E. Hanchak, R. Dye, C. J. Whitt, T. Davis, C. Birchfield, and J. E. Horton, "Polygraph and EEG correlates of lie detection," Psychophysiology, vol. 45, pp. S109-S109, 2008.
  8. T. M. C. Lee, H. L. Liu, L. H. Tan, C. C. H. Chan, S. Mahankali, C. M. Feng, J. W. Hou, P. T. Fox, and J. H. Gao, "Lie detection by functional magnetic resonance imaging," Hum. Brain Mapp., vol. 15, pp. 157-164, 2002. https://doi.org/10.1002/hbm.10020
  9. A. M. Owen, and M. R. Coleman, "Detecting awareness in the vegetative state," Molecular and Biophysical Mechanisms of Arousal, Alertness, and Attention (Oxford: Blackwell Publishing), vol. 1129, pp. 130-138, 2008.
  10. A. M. Owen, M. R. Coleman, M. Boly, M. H. Davis, S. Laureys, and J. D. Pickard, "Using functional magnetic resonance imaging to detect covert awareness in the vegetative state," Arch. Neurol., vol. 64, pp. 1098-1102, 2007. https://doi.org/10.1001/archneur.64.8.1098
  11. A. S. Royer, and B. He, "Goal selection versus process control in a brain-computer interface based on sensorimotor rhythms," J. Neural Eng., vol. 6, 2009.
  12. M. Schreuder, B. Blankertz, and M. Tangermann, "A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue," Plos One, vol. 5, 2010.
  13. R. Stark, W. Ambach, J. Lange, E. Bauer, and D. Vaitl, "Liedetection in a mock crime scenario: Insights by functional magnetic resonance imaging," J. Psychophysiol., vol. 20, pp. 115-115, 2006.
  14. J. R. Binder, J. A. Frost, T. A. Hammeke, P. S. F. Bellgowan, S. M. Rao, and R. W. Cox, "Conceptual processing during the conscious resting state: A functional MRI study," J. Cogn. Neurosci., vol. 11, pp. 80-93, 1999. https://doi.org/10.1162/089892999563265
  15. D. Chawla, G. Rees, and K. J. Friston, "The physiological basis of attentionl modulation in extrastriate visual areas," Nat. Neurosci., vol. 2, pp. 671-676, 1999. https://doi.org/10.1038/10230
  16. K. M. O'Craven, and N. Kanwisher, "Mental imagery of faces and places activates corresponding stimulus-specific brain regions," J. Cogn. Neurosci., vol. 12, pp. 1013-1023, 2000. https://doi.org/10.1162/08989290051137549
  17. N. Kriegeskorte, R. Goebel, and P. Bandettini, "Informationbased functional brain mapping," Proc. Natl. Acad. Sci. U. S. A., vol. 103, pp. 3863-8, 2006. https://doi.org/10.1073/pnas.0600244103
  18. K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby, "Beyond mind-reading: multi-voxel pattern analysis of fMRI data," Trends Cogn. Sci., vol. 10, pp. 424-430, 2006. https://doi.org/10.1016/j.tics.2006.07.005
  19. J. V. Haxby, M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, and P. Pietrini, "Distributed and overlapping representations of faces and objects in ventral temporal cortex," Science, vol. 293, pp. 2425-30, 2001. https://doi.org/10.1126/science.1063736
  20. J. D. Haynes, and G. Rees, "Predicting the orientation of invisible stimuli from activity in human primary visual cortex," Nat. Neurosci., vol. 8, pp. 686-91, 2005. https://doi.org/10.1038/nn1445
  21. S. M. Polyn, V. S. Natu, J. D. Cohen, and K. A. Norman, "Category-specific cortical activity precedes retrieval during memory search," Science, vol. 310, pp. 1963-6, 2005. https://doi.org/10.1126/science.1117645
  22. M. Spiridon, and N. Kanwisher, "How distributed is visual category information in human occipito-temporal cortex? An fMRI study," Neuron, vol. 35, pp. 1157-1165, 2002. https://doi.org/10.1016/S0896-6273(02)00877-2
  23. A. S. Gevins, G. M. Zeitlin, J. C. Doyle, R. E. Schaffer, and E. Callaway, "EEG patterns during 'cognitive' tasks. II. Analysis of controlled tasks," Electroencephalogr. Clin. Neurophysiol., vol. 47, pp. 704-710, 1979. https://doi.org/10.1016/0013-4694(79)90297-9
  24. A. S. Gevins, G. M. Zeitlin, C. D. Yingling, J. C. Doyle, M. F. Dedon, R. E. Schaffer, J. T. Roumasset, and C. L. Yeager, "EEG patterns during 'cognitive' tasks. I. Methodology and analysis of complex behaviors," Electroencephalogr. Clin. Neurophysiol., vol. 47, pp. 693-703, 1979. https://doi.org/10.1016/0013-4694(79)90296-7
  25. Z. A. Keirn, and J. I. Aunon, "A new mode of communication between man and his surroundings," IEEE Trans. Biomed. Eng., vol. 37, pp. 1209-14, 1990. https://doi.org/10.1109/10.64464
  26. B. D. Mensh, J. Werfel, and H. S. Seung, "BCI competition 2003 - Data set Ia: Combining gamma-band power with slow cortical potentials to improve single-trrial classification of electroencephalographic signals," IEEE Trans. Biomed. Eng., vol. 51, pp. 1052-1056, 2004. https://doi.org/10.1109/TBME.2004.827081
  27. R. Palaniappan, "Utilizing gamma band to improve mental task based brain-computer Interface design," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 14, pp. 299-303, 2006. https://doi.org/10.1109/TNSRE.2006.881539
  28. W. D. Penny, S. J. Roberts, E. A. Curran, and M. J. Stokes, "EEG-Based communication: A pattern recognition approach," IEEE Trans. Rehabil. Eng., vol. 8, pp. 214-215, 2000. https://doi.org/10.1109/86.847820
  29. L. Zhang, W. He, C. H. He, and P. Wang, "Improving Mental Task Classification by Adding High Frequency Band Information," J. Med. Syst., vol. 34, pp. 51-60, 2010. https://doi.org/10.1007/s10916-008-9215-z
  30. A. F. Cabrera, and K. Dremstrup, "Auditory and spatial navigation imagery in Brain-Computer Interface using optimized wavelets," J. Neurosci. Methods, vol. 174, pp. 135-146, 2008. https://doi.org/10.1016/j.jneumeth.2008.06.026
  31. A. F. Cabrera, D. Farina, and K. Dremstrup, "Comparison of feature selection and classification methods for a brain-computer interface driven by non-motor imagery," Med. Biol. Eng. Comput., vol. 48, pp. 123-132, 2010. https://doi.org/10.1007/s11517-009-0569-2
  32. H. J. Hwang, K. H. Kim, Y. J. Jung, D. W. Kim, Y. H. Lee, and C. H. Im, "An EEG-based real-time cortical functional connectivity imaging system," Med. Biol. Eng. Comput., vol. 49, pp. 985-995, 2011. https://doi.org/10.1007/s11517-011-0791-6
  33. J. E. Desmedt, and C. Tomberg, "Transient phase-locking of 40 Hz electrical oscillations in prefrontal and parietal human cortex reflects the process of conscious somatic perception," Neurosci. Lett., vol. 168, pp. 126-129, 1994. https://doi.org/10.1016/0304-3940(94)90432-4
  34. S. P. Fitzgibbon, K. J. Pope, L. Mackenzie, C. R. Clark, and J. O. Willoughby, "Cognitive tasks augment gamma EEG power," Clin. Neurophysiol., vol. 115, pp. 1802-1809, 2004. https://doi.org/10.1016/j.clinph.2004.03.009
  35. D. W. Gross, and J. Gotman, "Correlation of high-frequency oscillations with the sleep-wake cycle and cognitive activity in humans," Neuroscience, vol. 94, pp. 1005-1018, 1999. https://doi.org/10.1016/S0306-4522(99)00343-7
  36. K. Sauve, "Gamma-Band Synchronous Oscillations: Recent Evidence Regarding Their Functional Significance," Conscious. Cogn., vol. 8, pp. 213-224, 1999. https://doi.org/10.1006/ccog.1999.0383
  37. Z. A. Keirn, and J. I. Aunon, "Man-machine communications through brain-wave processing," Engineering in Medicine and Biology Magazine, IEEE, vol. 9, pp. 55-57, 1990. https://doi.org/10.1109/51.62907
  38. G. Pfurtscheller, C. Brunner, A. Schlogl, and F. H. L. da Silva, "Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks," Neuroimage, vol. 31, pp. 153-159, 2006. https://doi.org/10.1016/j.neuroimage.2005.12.003
  39. J. H. Lee, M. Marzelli, F. A. Jolesz, and S. S. Yoo, "Automated classification of fMRI data employing trial-based imagery tasks," Med. Image Anal., vol. 13, pp. 392-404, 2009. https://doi.org/10.1016/j.media.2009.01.001
  40. H. J. Hwang, K. Kwon, and C. H. Im, "Neurofeedback-based motor imagery training for brain-computer interface (BCI)," J. Neurosci. Methods, vol. 179, pp. 150-156, 2009. https://doi.org/10.1016/j.jneumeth.2009.01.015
  41. P. Pudil, J. Novovicová, and J. Kittler, "Floating search methods in feature selection," Pattern Recogn. Lett., vol. 15, pp. 1119-1125, 1994. https://doi.org/10.1016/0167-8655(94)90127-9
  42. R. Ishii, K. Shinosaki, S. Ukai, T. Inouye, T. Ishihara, T. Yoshimine, N. Hirabuki, H. Asada, T. Kihara, S. E. Robinson, and M. Takeda, "Medial prefrontal cortex generates frontal midline theta rhythm," Neuroreport, vol. 10, pp. 675-679, 1999. https://doi.org/10.1097/00001756-199903170-00003