DOI QR코드

DOI QR Code

Web Document Classification Based on Hangeul Morpheme and Keyword Analyses

한글 형태소 및 키워드 분석에 기반한 웹 문서 분류

  • 박단호 (한국외국어대학교 산업경영공학과) ;
  • 최원식 (한국외국어대학교 산업경영공학과) ;
  • 김홍조 (한국외국어대학교 산업경영공학과) ;
  • 이석룡 (한국외국어대학교 산업경영공학과)
  • Received : 2012.01.09
  • Accepted : 2012.06.20
  • Published : 2012.08.31

Abstract

With the current development of high speed Internet and massive database technology, the amount of web documents increases rapidly, and thus, classifying those documents automatically is getting important. In this study, we propose an effective method to extract document features based on Hangeul morpheme and keyword analyses, and to classify non-structured documents automatically by predicting subjects of those documents. To extract document features, first, we select terms using a morpheme analyzer, form the keyword set based on term frequency and subject-discriminating power, and perform the scoring for each keyword using the discriminating power. Then, we generate the classification model by utilizing the commercial software that implements the decision tree, neural network, and SVM(support vector machine). Experimental results show that the proposed feature extraction method has achieved considerable performance, i.e., average precision 0.90 and recall 0.84 in case of the decision tree, in classifying the web documents by subjects.

Acknowledgement

Supported by : 한국외국어대학교

References

  1. K.Y. Sung and B.H. Yun, "Topic based Web Document Clustering using Named Entities," Journal of Korean Contents, Vol.10, No.5, 2010, pp.29-36. https://doi.org/10.5392/JKCA.2010.10.5.029
  2. W.S. Bae, Y. S. Han, and J. W. Cha, "Text Categorization using Topic Signature and Co-occurrence Features," Proc. of KCC2008, Vol.35, No.1, 2008, pp.1-8.
  3. E.K. Chung, "A Semantic-Based Feature Expansion Approach for Improving the Effectiveness of Text Categorization by Using WordNet," Journal of the Korean Society for information Management , Vol.26, No.3, 2009, pp.261-278. https://doi.org/10.3743/KOSIM.2009.26.3.261
  4. Y. Yang and J. Pedersen, "A Comparative Study on Feature Selection in Text Categorization," Proc. of 14th Int. Conf. on Machine Learning, 1997, pp.412-420.
  5. Forman, G., "An extensive empirical study of feature selection metrics for text classification," J. Mach. Learn. Res., Vol.3 (2003), pp.1289-1305.
  6. Y.J. Nam and K.H. Kim, "A Study on Automatic Text Categorization of Web-Based Query Using Synonymy List," Journal of information management, Vol.35, No.4, 2004, pp.81-105. https://doi.org/10.1633/JIM.2004.35.4.081
  7. Y.J. Nam, "A study of Korean automatic indexing by morphological analysis," Ph.D Thesis, Chungang University, Seoul, 1995.
  8. D. Ludovic G. Patrick, and Z. Hugo, "HMM-based Passage Models for Document Classification and Ranking," 23rd BCS European Annual Colloquium on Info. Retrieval, 2001.
  9. W. Chen, X. Chang, H. Wang, J. Zhu, and T. Yao, "Automatic Word Clustering for Text Categorization Using Global Information," LNCS Vol.3411, 2005, pp.1-11.
  10. T. Theeramunkong, "Applying passage in Web text mining," Int. J of Intelligent Systems - Intelligent Technologies, Vol.19, Issue 1-2, 2004, pp.149-158. https://doi.org/10.1002/int.10158
  11. http://www.cs.cmu.edu/-mccallum/bow /rainbow/
  12. J.S. Lee and J.W. Lee, "A Hangeul Document Classification System using Case-based Reasoning," Asia Pacific Journal of Information Systems, Vol.12, No.2, 2002, pp.179-195.
  13. S.S. Kang, KLT version 2.2.0., http://nlp.kookmin.ac.kr, Korean Language Processing and Information Retrieval Laboratory, 2010.

Cited by

  1. Study on the Trends of U-City and Smart City Researches using Text Mining Technology vol.22, pp.3, 2014, https://doi.org/10.7319/kogsis.2014.22.3.087
  2. Website Classification based on Occurrence Frequency of Medical Terms and Hyperlinks in Webpage vol.23, pp.2, 2013, https://doi.org/10.5391/JKIIS.2013.23.2.126
  3. Automatic Classification of Blog Posts using Various Term Weighting vol.39, pp.1, 2015, https://doi.org/10.5916/jkosme.2015.39.1.58