Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Effect of Application of Ensemble Method on Machine Learning with Insufficient Training Set in Developing Automated English Essay Scoring System

영작문 자동채점 시스템 개발에서 학습데이터 부족 문제 해결을 위한 앙상블 기법 적용의 효과

  • 이경호 (충남대학교 전자전파정보통신공학과) ;
  • 이공주 (충남대학교 전자전파정보통신공학과)
  • Received : 2015.04.14
  • Accepted : 2015.07.21
  • Published : 2015.09.15
⟨ Previous Next ⟩

Abstract

In order to train a supervised machine learning algorithm, it is necessary to have non-biased labels and a sufficient amount of training data. However, it is difficult to collect the required non-biased labels and a sufficient amount of training data to develop an automatic English Composition scoring system. In addition, an English writing assessment is carried out using a multi-faceted evaluation of the overall level of the answer. Therefore, it is difficult to choose an appropriate machine learning algorithm for such work. In this paper, we show that it is possible to alleviate these problems through ensemble learning. The results of the experiment indicate that the ensemble technique exhibited an overall performance that was better than that of other algorithms.

일반적으로, 교사 학습 알고리즘이 적절히 학습되기 위해서는 레이블의 편향이 없는 충분한 양의 학습데이터가 필요하다. 그러나 영작문 자동채점 시스템 개발을 위한 충분하고 편향되지 않은 학습데이터를 수집하는 것은 어려운 일이다. 또한 영어 작문 평가의 경우, 전체적인 답안 수준에 대한 다면적인 평가가 이루어진다. 적고 편향되기 쉬운 학습데이터와 이를 이용한 여러 평가영역에 대한 학습모델을 생성해야하기 때문에, 이를 위한 적절한 기계학습 알고리즘을 결정하기 어렵다. 본 논문에서는 이러한 문제를 앙상블학습을 통해 완화할 수 있음을 실험에 통해 보이고자 한다. 실제 중, 고등학교 학생들을 대상으로 시행된 단문형 영작문 채점 결과를 학습데이터 개수와 편향성을 조절하여 실험하였다. 학습데이터의 개수 변화와 편향성 변화의 실험 결과, 에이다부스트 알고리즘을 적용한 결과를 투표로 결합한 앙상블 기법이 다른 알고리즘들 보다 전반적으로 더 나은 성능을 나타냄을 실험을 통해 나타내었다.

Keywords

Acknowledgement

Supported by : 한국교육과정평가원

References

  1. G.H Lee and K.J Lee, "Developing an Automated English Sentence Scoring System for Middleschool Level Writing Test by Using Machine Learning Techniques," Journal of KIISE, Vol. 41, No. 11, pp. 911-920, 2014. https://doi.org/10.5626/JOK.2014.41.11.911
  2. J. E. Kim and K. J. Lee, "Implementing Automated English Error Detecting and Scoring System for Junior High School Students," The Journal of the Korea Contents Association, Vol. 7, No. 5, pp. 36-46, 2007. https://doi.org/10.5392/JKCA.2007.7.5.036
  3. Attali, Yigal, and Jill Burstein, "Automated essay scoring with $e-rater^{(R)}$ V. 2." The Journal of Technology, Learning and Assessment, Vol. 4, No. 3, 2006.
  4. He, Haibo, and Edwardo A. Garcia, "Learning from imbalanced data," Knowledge and Data Engineering, IEEE Transactions, Vol. 21, No. 9, pp. 1263-1284, 2009. https://doi.org/10.1109/TKDE.2008.239
  5. KESS, "2014 Basic Education Statistics," 2014. (in korean)
  6. Dikli, Semire, "An overview of automated scoring of essays," The Journal of Technology, Learning and Assessment, Vol. 5, No. 1, 2006.
  7. Freund, Yoav, and Robert E. Schapire, "A desiciontheoretic generalization of on-line learning and an application to boosting," Computational learning theory, Springer Berlin Heidelberg, 1995.
  8. Wolpert, David H., "Stacked generalization," Neural networks, Vol. 5, No. 2, pp. 241-259, 1992. https://doi.org/10.1016/S0893-6080(05)80023-1
  9. Bin, Li, and Yao Jian-Min, "Automated essay scoring using multi-classifier fusion," Computing and Intelligent Systems, Springer Berlin Heidelberg, pp. 151-157, 2011.
  10. Cortes, Corinna, and Vladimir Vapnik. "Supportvector networks," Machine Learning, Vol. 20, No. 3, pp. 273-297, 1995. https://doi.org/10.1007/BF00994018
  11. Haykin, S. Neural Networks: A Comprehensive Foundation, Prentice Hall. 2ed. 1998.
  12. Murphy, Kevin P., "Naive bayes classifiers," University of British Columbia, 2006.
  13. Colas, Fabrice, and Pavel Brazdil, "Comparison of SVM and some older classification algorithms in text classification tasks," Artificial Intelligence in Theory and Practice, Springer US, pp. 169-178, 2006.
  14. Ting, S. L., W. H. Ip, and Albert HC Tsang, "Is Naive Bayes a good classifier for document classification?," International Journal of Software Engineering and Its Applications, Vol. 5, No. 3, pp. 37.46, 2011.