In an experiment where the true KL divergence between two normal distributions was 0.5, the biase..., Sonic AI
“In an experiment where the true KL divergence between two normal distributions was 0.5, the biased estimator `1/2 * (log(r))^2` had a bias of 25% of the true value.”