Online Learning of Bayesian Network Parameters for Incomplete Data of Real World

현실 세계의 불완전한 데이타를 위한 베이지안 네트워크 파라메터의 온라인 학습

  • Published : 2006.12.15

Abstract

The Bayesian network(BN) has emerged in recent years as a powerful technique for handling uncertainty iii complex domains. Parameter learning of BN to find the most proper network from given data set has been investigated to decrease the time and effort for designing BN. Off-line learning needs much time and effort to gather the enough data and since there are uncertainties in real world, it is hard to get the complete data. In this paper, we propose an online learning method of Bayesian network parameters from incomplete data. It provides higher flexibility through learning from incomplete data and higher adaptability on environments through online learning. The results of comparison with Voting EM algorithm proposed by Cohen at el. confirm that the proposed method has the same performance in complete data set and higher performance in incomplete data set, comparing with Voting EM algorithm.

최근 현실 세계의 불확실한 환경을 극복하기 위한 방법 중 하나로 베이지안 네트워크(Bayesian network, BN)가 부각되고 있다. BN의 파라메터 학습은 주어진 평가 척도에 따라 데이타의 훈련집합에 가장 잘 부합되는 네트워크 파라메터를 구하는 것으로, BN 설계에 드는 시간과 노력을 줄이기 위해 연구되어 왔다. 기존의 오프라인 학습은 학습에 필요한 충분한 양의 데이타를 모으기에는 많은 노력과 시간이 필요하다. 또한 현실세계는 불완전성을 포함하고 있어 완전한 데이타를 얻기 힘들다. 본 논문에서는 불완전한 데이타로부터 온라인으로 BN 파라메터를 학습하는 방법을 제안한다. 이 방법은 불완전한 데이타로부터 학습이 가능하도록 하여 학습의 유연성을 높이고, 실시간 학습을 통해 변화하는 환경에 대한 적응성을 높인다. Cohen 등이 제안한 온라인 파라메터 학습방법인 Voting EM 알고리즘과 비교 실험한 결과, 완전한 데이타를 가지고 학습한 경우에는 동일한 학습 결과를, 그리고 불완전한 데이타의 경우에는 보다 나은 학습 결과를 얻었다.

Keywords

References

  1. D. Heckeman, 'Bayesian networks for data mining,' Data Mining and Knowledge Discovery, vol. 1, pp. 79-119, 1997 https://doi.org/10.1023/A:1009730122752
  2. H.G. Cooper and E. Herskovitz, 'A Bayesian method for the induction of probabilistic networks from data,' Machine Learning, vol. 9, pp. 159-225, 1994 https://doi.org/10.1023/A:1022649401552
  3. W. L. Bunine, 'Operations for learning with graphical models,' Journal of Artificial Intelligence Research, vol. 2, pp. 159-225, 1994
  4. D. Heckerman, D. Geiger and D.M. Chickering, 'Learning Bayesian networks: The combinations of knowledge and statistical data,' Machine Learning, vol. 20, pp. 197-243, 1995 https://doi.org/10.1007/BF00994016
  5. R.G. Cowell, A.P. Dawid and P. Sebastiani, 'A comparison of sequential learning methods for incomplete data,' In Bayesian Statistics, vol. 5, pp. 533-542, 1996
  6. A. Dempster, D. Laird and D. Rubin, 'Maximum likelihood from incomplete data via the EM algorithm,' Journal of the Royal Statistical Society, Series B, vol. 39, pp. 1-38, 1977
  7. S. Geman and D. Geman, 'Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, pp. 721-741, 1984 https://doi.org/10.1109/TPAMI.1984.4767596
  8. D. Spiegelhalter and S. Lauritzen, 'Sequential updating of conditional probabilities on directed graphical structures,' Networks, vol. 20, pp. 579-605, 1990 https://doi.org/10.1002/net.3230200507
  9. E. Bauer, D. Koller and Y. Singer, 'Update rules for parameter estimation in Bayesian networks,' Proceedings of the 13th. Annual Conference on Uncertainty in AI, pp. 3-13, 1997
  10. I. Cohen, A. Bronstein and F.G. Cozman, 'Online learning of Bayesian network parameters,' In Report No. HPL-2001-55, HP Labs, 2001
  11. I. Cohen, A. Bronstein and F.G. Cozman, 'Adaptive online learning of Bayesian network parameters,' In Report No. HPL-2001-156, HP Labs, 2001
  12. S.Z. Zhang, H. Yu, H. Ding, N.H. Yang and X.K. Wang, 'An application of online learning algorithm for Bayesian network parameter,' Machine Lear-ning and Cybernetics, vol. 1, pp. 153-156, 2003
  13. S.L. Lauritzen and D.J. Spiegelhalter, 'Local com-putations with probabilities on graphical structures and their application to expert systems (with discussion),' Journal of the Royal Statistical Society, Series B, vol. 50, pp. 157-224, 1988
  14. B. D'Ambrosio, 'Inference in Bayesian networks,' AI Magazine, vol. 20, no. 2, pp. 21-36, 1999