Journal of the Korean Data and Information Science Society

한국데이터정보과학회 (The Korean Data and Information Science Society)
권호 표지
DOI QR코드

DOI QR Code

태권도 품새 경기의 주관적 평가결과의 오차원 분석: 일반화가능도 이론 적용

Analysis of error source in subjective evaluation results on Taekwondo Poomsae: Application of generalizability theory

  • 조은형 (성균관대학교 체력과학연구소)
  • Cho, Eun Hyung (The Research Institution for Physical Fitness and Sport Science, Sungkyunkwan University)
  • 투고 : 2016.01.04
  • 심사 : 2016.03.07
  • 발행 : 2016.03.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 연구는 G-Theory를 적용하여 태권도 품새 평가항목에 대한 채점자 간 평가점수의 신뢰도를 추정하기 위해 G-연구로 경기일 수, 채점자 수를 다중오차원으로 선정하고 이들 변인간의 상호작용으로 인한 오차변량의 상대적 크기에 의해 오차원을 분석하고, G-연구의 결과를 토대로 D-연구를 수행하여 최적의 측정조건을 결정하는 데 목적이 있다. 결과를 요약하면 다음과 같다. G-Theory를 적용하여 태권도 품새 평가항목 중 정확성에 대해 분산성분 추정치를 추정한 결과, 오차의 영향력은 채점자국면에서 가장 크게 나타났으며, 각 설계에 따른 상호효과 (피험자 내), 피험자 간 순서였으며, 표현성 평가항목에 대한 분산성분 추정치 오차의 영향력은 상호효과 (피험자 내)국면에서 가장 크게 나타났으며, 각 설계에 따른 피험자 간, 채점자 국면 순으로 나타났다. 마지막으로 D-연구를 통하여 일반화가능도계수를 추정한 결과, 채점자 수에 따른 최적수준의 측정조건은 정확성 평가항목에서 8명의 채점자일 때 안정적인 신뢰도를 얻을 수 있으며, 표현성 평가항목에서는 7명의 채점자일 때 안정적인 신뢰도를 얻었다.

This study aims to apply the G-theory for estimation of reliability of evaluation scores between raters on Taekwondo Poomsae rating categories. Selecting a number of game days and raters as multiple error sources, we analyzed the error sources caused by relative magnitude of error variances of interaction between the factors and proceeded with D-study based on the results of G-study for optimal determination of measurement condition. The results showed below. The estimated outcomes of variance component for accuracy among the Taekwondo Poomsae categories with G-theory showed that impact of error was the biggest influence factor in raters conditions and in order of interaction in subjects and between subjects, also impact of variance component estimation error on expression category was the major influence factor in interaction and in order of the between subjects and raters. Finally, the result of generalizability coefficient estimation via D-study showed that measurement condition of optimal level depend on the number of raters was 8 persons of raters on accuracy category, and stable reliability on expression category was gained when the raters were 7 persons.

키워드

참고문헌

  1. Ahn, A. J. (1997). Application of generalizability in analyzing sources of error variance of gymnastic judgement, Ph. D. Thesis, Myungji University, Kyeonggido.
  2. Allen, M. J. and Yen, W. M. (1979). Introduction to measurement theory, 1st ed., Waveland Press, Illinois.
  3. Brennan, R. L. (1983). Elements of generalizability theory, ACT Publication, Iowa.
  4. Cho, E. H. (2014). Development of computer-based assessment instrument and movement classification for poomsae skills in taekwondo, Ph. D. Thesis, Sungkyunkwan University, Kyeonggido.
  5. Cho, E. H., Kwon, J. H., Cha, C. H. and Kim, K. S. (2005). The severity and consistency of the judge's decision in taekwondo. Journal of Sport Leisure Studies, 25, 85-95.
  6. Cho, J. H. (1991). Genralizability coefficient estimation of skinfold thickness test and basketball skill test, Ph. D. Thesis, Korean National College of Physical Education, Seoul.
  7. Cho, J. H. (2006). An analysis of sources of error and optimal condition of generalizability coefficient in the Korean national bowling player selective match. The Korean Journal of Physical Education, 45, 457-167.
  8. Choi, J. S. (2014). Projection analysis for two-way variance components. Journal of the Korean Data & Information Science Society, 25, 547-554. https://doi.org/10.7465/jkdi.2014.25.3.547
  9. Choi, Y. L. (2013). The olympic experience and expertise of taekwondo national representative coach, Ph. D. Thesis, Korea National Sport University, Seoul.
  10. Cronbach, L. J., Gleser, G. C., Nanda, H. and Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalization for scores and profiles, Wiley, New York.
  11. Hagtvet, K.nut A. (1997). The function of indicators and errors in construct measures: An application of generalizability theory. Journal of Vocational Education Research, 22, 247-266.
  12. Heo, C. K. (2003) Generalizability coefficient estimation of subjectivity volleyball test, Ph. D. Thesis, Kookmin University, Seoul.
  13. Jung, Y. H. (2009). Analysis on reliability of referees judgment in Taekwondo competition applying generalizbility theory, Ph. D. Thesis, Kookmin University, Seoul.
  14. Kang, S. J. and Ann, A. J. (1999). Rater's objectivity in the assessment of gymnastic performance. The Korean Journal of Physical Education, 38, 641-650.
  15. KERIS (2015). http://www.riss.kr.
  16. Kim, D. Y. and Heo, C. K. (2002). Generalizability coefficient estimation of subjectively volleyball test. The Korean Journal of Measurement and Evaluation in Physical Education and Sport Science, 4, 15-28.
  17. Kim, M. Y. and Kang, S. J. (2003). Examining rater errors in performance assessment with a many-faceted rasch model. The Korean Journal of Physical Education, 42, 1019-1027.
  18. Kim, S. S. (1989). An analysis of sources of variation in teacher behaviors using generalizability theory, Ph. D. Thesis, University of Virginia, Virginia.
  19. Kim, S. S. (1993). The generalizability of student ratings of instructors across sections. The Korean Journal of Educational Research, 31, 23-40.
  20. Kim, Y. B. and Kim, S. S. (2001). Generalizability theory, Kyohaksa of Publisher, Seoul.
  21. Ko, B. G. (1997). A Generalizability of physical performance tests for apparatus exercises in elementary school. Journal of the Research Institute of Physical Education, 18, 111-126.
  22. Kukkiwon (2015). The countries geups & dans situation, Retrieved March 5, 2015, from http://www.kukkiwon.or.kr.
  23. Lee, J. T. (2010). A Study on the performance pattern in the elite decathlon. Journal of the Korean Data & Information Science Society, 21, 1071-1079.
  24. Lee, J. Y. (2005). Estimating the reliability of the selection method of the national team bowlers; The application of generalizability theory, Ph. D. Thesis, Korea National Sport University, Seoul.
  25. Looney, M. (1997). Objectivity measurement of figure skating performance. Journal of outcome measurement, 1, 143-163.
  26. MacMillan, P. D. (2000). Classical, generalizability, and multifaceted rasch detection of interrater variability in large, sparse data sets. The Journal of Experimental Education, 68, 167-190. https://doi.org/10.1080/00220970009598501
  27. Nunally, J. C. (1978). Psychometric theory, 2nd ed., McGraw-Hill, New York.
  28. Park, H, C. (2014). Comparison of confidence measures useful for classification model building. Journal of the Korean Data & Information Science Society, 25, 365-371. https://doi.org/10.7465/jkdi.2014.25.2.365
  29. Park, H. J. and Pi, S. Y. (2013). Study on the K-scale reflecting the confidence of survey responses. Journal of the Korean Data & Information Science Society, 24, 41-51. https://doi.org/10.7465/jkdi.2013.24.1.41
  30. Safrit, M. J., Atwater, A. E., Baumgarther, T. A andWest, C. (1976). Reliability theory, American Alliance for Health, Physical Education and Recreation, Washington.
  31. Shavelson, R. J and Webb, N. M. (1981). Generalizability theory: 1973-1980. British Journal of Mathematical and Statistical Psychology, 34, 133-166. https://doi.org/10.1111/j.2044-8317.1981.tb00625.x
  32. Shavelson, R. J., Baxter. G. P. and Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30, 215-232. https://doi.org/10.1111/j.1745-3984.1993.tb00424.x
  33. Sun, A., Michael J. V. and Gao, X. (1997). Using generalizability theory to assess the reliability of student ratings of academic advising. The Journal of Experimental Education, 65, 367-379. https://doi.org/10.1080/00220973.1997.10806611
  34. WTF (2014). http://www.wtf.org.
  35. Zhu, W., Ennis, C. D. and Chin, A. (1998). Many faceted rasch modeling expert judgement in test development. Measurement in physical education and exercise science, 2, 21-39. https://doi.org/10.1207/s15327841mpee0201_2

피인용 문헌

  1. Analysis of objective and error source of ski technical championship vol.28, pp.4, 2016, https://doi.org/10.24985/kjss.2017.28.4.1083