Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지

A Comparison on the Differential Entropy

  • Kim, Dae-Hak (Catholic University of Daegu, Department of Statistical information)
  • Published : 2005.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Entropy is the basic concept of information theory. It is well defined for random varibles with known probability density function(pdf). For given data with unknown pdf, entropy should be estimated. Usually, estimation of entropy is based on the approximations. In this paper, we consider a kernel based approximation and compare it to the cumulant approximation method for several distributions. Monte carlo simulation for various sample size is conducted.

Keywords

References

  1. Biometrika v.71 An alternative method of cross-validation for the smoothing of density estimates Bowman, A.
  2. Elements of Information theory Cover, T.M.;Thomas, J.A.
  3. Independent Component Analysis Hyvarinen, A.;Karhunen, J.;Oja, E.
  4. Probability, Random variables and Stochastic Process Papoulis, A.
  5. Density estimation for statistics and data analysis Silverman, B.W.