Inter-rater Reliability and Training Effect of the Differential Diagnosis of Speech and Language Disorder for Stroke Patients

뇌졸중 환자의 말, 언어장애 선별에 대한 검사자간 신뢰도 및 훈련효과

  • 김정완 (부산가톨릭대학교 언어청각치료학과)
  • Received : 2010.10.07
  • Accepted : 2011.08.08
  • Published : 2011.09.28


Distinguishing aphasia in stroke patients and observing the subtle linguistic characteristics associated with it primarily requires the use of instruments that provide reliable assessment results. Additionally, examiners should be fully aware of how to use those instruments. This study examined 46 stroke patients for aphasia and assessed the reliability of their diagnoses according to examiners whose medical fields were different from each other. Furthermore, a comparison was made between the reliability before training and that after training. To this end, 46 stroke patients were tested for aphasia and in terms of their speech disorder degree by 3 groups, each of which consisted of 12 professionals (3 SLP, 3 neurologist, and 3 nurse). In the result, a rating of 'acceptable' was given for speech intelligibility tasks and the voice quality of /ah-/ prolongation, and other sub-tests were marked as 'good-excellent' by the experts with different areas of medical expertise. For the tasks marked as 'acceptable', the subjects were video-trained for 3 weeks and the differences were compared before and after their training. Consequently, the differences in the examiners' ratings in the speech intelligibility tasks showed a significant decrease and the accuracy of their voice quality ratings showed a significant increase. In the result of research on the correlation between the accuracy of the sub-test ratings and the amount of clinic experience, speech therapists developed more accuracy in rating a picture description task and a speech intelligibility task as their experience accumulated. Meanwhile, doctors and nurses showed more accurate ratings in picture description tasks with greater clinical experience. The results of this study suggest that assessing the neurologic-communicative disorders of stroke patients requires ongoing training and experience, especially for speech disorders. It was also found that the rating reliability in this case could be improved by training.


Aphasia Screening;Inter-rater Reliability;Stroke;Training


  1. P. T. Lyden, B. Brott, K. M. Tilley, E. J. Welch, S. Mascha, E. C. Levine, EC. Haley, J. Grotta, and J. Marler, NINDS TPA Stroke Study Group, "Improved reliability of the NIH Stroke Scale using video training," Stroke, Vol.25, No.11, pp.2220-2226, 1994.
  2. J. Kreiman, B. R. Gerratt, G. B. Kempster, A. Erman, and G. S. Berke, "Perceptual Evaluation of Voice Quality: Review, Tutorial, and a Framework for Future Research," Journal of Speech and Hearing Research, Vol.36, pp.21-40, 1993.
  3. P. M. Pedersen, H. S. Jorgensen, H. Nakayama, O. H. Raaschou, and T. S. Olsen, "Aphasia in acute stroke: incidence, determinants and recovery," Annals of Neurology, Vol.38, No.4, pp.659-666, 1995.
  4. S. T. Engelter, M. Gostynski, S. Papa, M. Frei, C. Born, V. Ajdacic-Gross, F. Gutzwiller, and P. A. Lyrer, "Epidemiology of aphasia attributable to first ischemic stroke: incidence, severity, fluency, etiology, and thrombolysis," Stroke, Vol.37, No.6, pp.1379-1384, 2006.
  5. R. D. Kent, Intelligibility in speech disorders. Philadelphia, PA: John Benjarmins, 1992.
  6. 김향희, 이미숙, 김선우, 이원용, "파킨슨증으로 인한 마비말장애의 청지각적 평가에 대한 신뢰도 연구", 음성과학, 제11권, 제4호, pp.129-141, 2004.
  7. K. M. K. Chan, and E. ML. Yiu, "The Effect of Anchors and Training on the Reliability of Perceptual Voice Evaluation," J. of Speech, Language, and Hearing Research, Vol.45, pp.111-126, 2002.
  8. L. B .Goldstein, C. Bartels, and J. N. Davis, "Interrater reliability of the NIH stroke scale," Archives of Neurology, Vol.46, pp.660-662, 1999.<660::AID-ANA17>3.0.CO;2-Q
  9. K. E. Lewis, T. L. Watterson, and S. M. Houghton, "The influence of listener experience and academic training on ratings of nasality," J. of Communication Disorders, Vol,36, No.1, pp.49-58, 2003.
  10. E. Laczi, J. E. Sussman, E. T. Stathopoulos, and J. Huber, "Perceptual evaluation of hypernasality compared to HONC measures: the role of experience," Cleft Palate-Craniofacial Journal, Vol.42, pp.202-211, 2005.
  11. 김향희, 허지회, 김덕용, 김정완, 실어증-신경언어장애 선별검사(Screening test for aphasia & neurologic-communication disorders), 서울: 학지사, 2009.
  12. C. F. Waltz, O. L. Strickland, and E. R. Lenz, Measurement in nursing research. F. A. Davis, Philadelphia, 1991.
  13. H. Kim, and D. L. Na, "Normative Data on the Korean Version of the Western Aphasia Battery", J. of Clinical and Experimental Neuropsychology, Vol.26, No.8, pp.1011-1020, 2004.
  14. 김향희, 김정완, 허지회, 김덕용, 성수진, "실어증 선별검사 개발을 위한 내용타당도 검증", 언어청각장애연구, 제13권, 제3호, pp.353-380, 2008.
  15. C. F. Waltz, O. L. Strickland, and E. R. Lenz, Measurement in nursing research. F. A. Davis, Philadelphia, 1991.
  16. K. M. Yorskston, and D. R. Beukelman, Intelligibility measurement as a tool in the clinical management of dysarthric speakers, In R. D. Kent (Ed.), Intelligibility in speech disorders pp.265-286, Amsterdam: JohnBenjamins, 1992.
  17. J. Fricke, Reliability of the Functional Independence Measure with Occupational Therapists, Australian Occupational Therapy Journal, Vol.40, No.1, pp.7-15, 1993.