Divergent Priors and Well Behaved Bayes Factors

Abstract

Bartlett’s paradox has been taken to imply that using improper priors
results in Bayes factors that are not well defined, preventing model comparison
in this case. We use well understood principles underlying what is already
common practice, to demonstrate that this implication is not true for some
improper priors, such as the Shrinkage prior due to Stein (1956). While
this result would appear to expand the class of priors that may be used for
computing posterior odds, we warn against the straightforward use of these
priors. Highlighting the role of the prior measure in the behaviour of Bayes
factors, we demonstrate pathologies in the prior measures for these improper
priors. Using this discussion, we then propose a method of employing such
priors by setting rules on the rate of diffusion of prior certainty.