This was part of Bayesian Statistics and Statistical Learning

Rate-Distortion Theoretical Views of Bayesian Learning Coefficients

Kazuho Watanabe, Toyohashi University of Technology

Monday, December 11, 2023


We consider two types of rate-distortion problems associated with Bayesian learning. Formulating a rate-distortion problem by the distortion measure defined by the pointwise regret of the model parameter and another rate-distortion problem by replacing the empirical expectation with the true expectation, we show the relationships between the asymptotic behaviors of the rate-distortion functions and the learning coefficients of Bayesian learning. Furthermore, the two rate-distortion bounds for a fixed prior derive a relationship between the expected regret and Gibbs generalization error.