Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Topic directed Web Spidering using Reinforcement Learning

강화학습을 이용한 주제별 웹 탐색

  • Published : 2005.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we presents HIGH-Q learning algorithm with reinforcement learning for more fast and exact topic-directed web spidering. The purpose of reinforcement learning is to maximize rewards from environment, an reinforcement learning agents learn by interacting with external environment through trial and error. We performed experiments that compared the proposed method using reinforcement learning with breath first search method for searching the web pages. In result, reinforcement learning method using future discounted rewards searched a small number of pages to find result pages.

본 논문에서는 특정 주제에 관한 웹 문서들을 더욱 빠르고 정확하게 탐색하기 위하여 강화학습을 이용한 HIGH-Q 학습 알고리즘을 제안한다. 강화학습의 목적은 환경으로부터 주어지는 보상(reward)을 최대화하는 것이며 강화학습 에이전트는 외부에 존재하는 환경과 시행착오를 통하여 상호작용하면서 학습한다. 제안한 알고리즘이 주어진 환경에서 빠르고 효율적임을 보이기 위하여 넓이 우선 탐색과 비교하는 실험을 수행하고 이를 평가하였다. 실험한 결과로부터 우리는 미래의 할인된 보상을 이용하는 강화학습 방법이 정답을 찾기 위한 탐색 페이지의 수를 줄여줌으로써 더욱 정확하고 빠른 검색을 수행할 수 있음을 알 수 있었다.

Keywords

References

  1. 박찬건, 양성봉, '강화 학습에서의 탐색과 이용의 균 형을 통한 범용적 온라인 Q-학습이 적용된 에이전트 의 구현,' 정보과학회 논문지(B), Vol. 30, No. 7, pp. 672-680, 2003
  2. 정태진, 장병탁, '강화 학습을 이용한 웹 정보 검색,' 정보과학회 제 28회 추계학술대회, Vol. 28, No. 2, pp. 94-96, 2001
  3. C. J. Watkins and P. Dayan, 'Technical note : QLearning,' Machine Learning, 8, pp .279-292, 1992
  4. F. Menczer, 'ARACHNID: Adaptive retrieval agents choosing heuristic neighborhoods for information discovery,' In proceedings of 14th International Conference on Machine Learning, pp. 227-235, 1997
  5. H. Lieberman, 'Letizia: An agent that assists web browsing,' In Proocedings of the International Joint Conference on Arti cial Intelligence (IJCAI95), pp. 924-929, 1995
  6. J. Boyan, D. Freitag, and T. Joachimas, 'A machine learning architecture for optimizing web search engines,' In proceedings of AAAI workshop on Internet-Based Information Systems, pp. 1-8, 1996
  7. J. Peng, and R. Williams, 'Incremental multi-step Q-learning,' Machine Learning, vol. 22, pp. 283- 290, 1996
  8. J. Rennie and A. McCallum, 'Using Reinforcement Learning to Spider the Web Efficiently,' In proceedings of the 16th International Conference on Machine Learning(ICML-99), pp. 335-343, 1999
  9. L. P. Kaelbling, 'Learning in Embedded System,' PhD thesis, Departmenr of Computer Science, Stanford University, 1990
  10. R. Dearden, N. Friedman and S. Russell, 'Bayesian Q-Learning,' In proceedings of AAA-98, 1989
  11. R. S. Sutton and A. G. Barto, Reinforcement Learning : An Introduction. The MIT Press, 1998
  12. S. B. Thrun, 'The role of exploration in learning control,' Handbook of Intelligent Control:Neural, Fussy and Adaptive Approaches. 1992
  13. T. Joachims, D. Freitag, and T. M. Mitchell. 'A WebWatcher: A Tour Guide for the World Wide Web,' In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI'97), pp. 770-777, 1997
  14. T. M. Mitchell, Machine Learning, McGraw-Hill, 1997
  15. M. Tan, Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proc. of the Tenth International Conf. on Machine Learning, pp. 330.337, 1993