hrcak mascot   Srce   HID

Original scientific paper

Shannon's differential entropy asymptotic analysis in a Bayesian problem

Mark Kelbert   ORCID icon orcid.org/0000-0002-3952-2012 ; Higher School of Economics, National Research University, Moscow, RF
Pavel Mozgunov ; Department of Mathematics, Swansea University, Singleton Park, Swansea, UK

Fulltext: english, pdf (159 KB) pages 219-228 downloads: 311* cite
APA 6th Edition
Kelbert, M. & Mozgunov, P. (2015). Shannon's differential entropy asymptotic analysis in a Bayesian problem. Mathematical Communications, 20 (2), 219-228. Retrieved from https://hrcak.srce.hr/149788
MLA 8th Edition
Kelbert, Mark and Pavel Mozgunov. "Shannon's differential entropy asymptotic analysis in a Bayesian problem." Mathematical Communications, vol. 20, no. 2, 2015, pp. 219-228. https://hrcak.srce.hr/149788. Accessed 24 Jun. 2021.
Chicago 17th Edition
Kelbert, Mark and Pavel Mozgunov. "Shannon's differential entropy asymptotic analysis in a Bayesian problem." Mathematical Communications 20, no. 2 (2015): 219-228. https://hrcak.srce.hr/149788
Harvard
Kelbert, M., and Mozgunov, P. (2015). 'Shannon's differential entropy asymptotic analysis in a Bayesian problem', Mathematical Communications, 20(2), pp. 219-228. Available at: https://hrcak.srce.hr/149788 (Accessed 24 June 2021)
Vancouver
Kelbert M, Mozgunov P. Shannon's differential entropy asymptotic analysis in a Bayesian problem. Mathematical Communications [Internet]. 2015 [cited 2021 June 24];20(2):219-228. Available from: https://hrcak.srce.hr/149788
IEEE
M. Kelbert and P. Mozgunov, "Shannon's differential entropy asymptotic analysis in a Bayesian problem", Mathematical Communications, vol.20, no. 2, pp. 219-228, 2015. [Online]. Available: https://hrcak.srce.hr/149788. [Accessed: 24 June 2021]

Abstracts
We consider a Bayesian problem of estimating of probability of success in a series of conditionally independent trials with binary outcomes. We study the asymptotic behaviour of differential entropy for posterior probability density function conditional on $x$ successes after $n$ conditionally independent trials, when $n \to \infty$. Three particular cases are studied: $x$ is a proportion of $n$; $x$ $\sim n^\beta$, where $0<\beta<1$; either $x$ or $n-x$ is a constant. It is shown that after an appropriate normalization in the first and second cases limiting distribution is Gaussian and the differential entropy of standardized RV converges to differential entropy of standard Gaussian random variable. In the third case the limiting distribution in not Gaussian, but still the asymptotic of differential entropy can be found explicitly.

Keywords
differential entropy; Bayes' formula; Gaussian limit theorem

Hrčak ID: 149788

URI
https://hrcak.srce.hr/149788

Visits: 471 *