Entropy bounds on Bayesian learning

Entropy bounds on Bayesian learning — Journal of Mathematical Economics, 44: 24-32, 2008
Authors

Olivier Gossner

Tristan Tomala

Abstract

An observer of a process (x_t) believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P(x_t|x1,…,x_{t−1}) and Q(x_t|x_1,…,x_{t−1}) for t = 1,…,n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.

← All research

Journal of Mathematical Economics, 44: 24-32, 2008

Paper PDF

Citation

BibTeX citation:
@article{gossner2008,
  author = {Gossner, Olivier and Tomala, Tristan},
  title = {Entropy Bounds on {Bayesian} Learning},
  journal = {Journal of Mathematical Economics},
  date = {2008},
  url = {https://gossner.me/papers/entropy-bounds-on-bayesian-learning.html},
  langid = {en},
  abstract = {An observer of a process (x\_t) believes the process is
    governed by Q whereas the true law is P. We bound the expected
    average distance between P(x\_t\textbar x1,…,x\_\{t−1\}) and
    Q(x\_t\textbar x\_1,…,x\_\{t−1\}) for t = 1,…,n by a function of the
    relative entropy between the marginals of P and Q on the n first
    realizations. We apply this bound to the cost of learning in
    sequential decision problems and to the merging of Q to P.}
}
For attribution, please cite this work as:
Gossner, Olivier, and Tristan Tomala. 2008. “Entropy Bounds on Bayesian Learning.” Journal of Mathematical Economics. https://gossner.me/papers/entropy-bounds-on-bayesian-learning.html.