New paper Detecting and diagnosing prior and likelihood sensitivity with power-scaling with @n-kall, @topipa, and @paul.buerkner ,

with Supplementary code

and priorsense R package.

Our goal was a prior sensitivity diagnostic that would be fully automatic for brms/rstanarm type packages or would require minimal user work otherwise. The prior and the likelihood are part of the same model, thus diagnosing the likelihood sensitivity makes sense, too.

We diagnose how much the posterior changes when prior or likelihood is power-scaled, i.e., exponentiated with some \alpha>0. This corresponds to scaling how informative the prior or the likelihood is.

Using importance sampling, we can easily compute arbitrary quantities measuring how much the posterior changes. As the default choice, we use the symmetrised metric version of cumulative Jensen-Shannon divergence.

For some quantities like mean and variance, we can also derive local analytic sensitivity at \alpha=1.

Ideally, the posterior is likelihood sensitive, i.e., the likelihood is informative. If the posterior is also prior sensitive, there might be a prior-likelihood conflict. If the posterior is only prior sensitive, there might non-informative likelihood.

We can provide the diagnostic for all parameters and interpretation of the diagnostic. All this is available in the priorsense R package.

The presence of prior sensitivity or the absence of likelihood sensitivity is not always a problem. Sometimes the prior can be strongly informative, and then a prior sensitivity can be desired.

Power-scaling prior and likelihood sensitivity diagnostic is not perfect and can miss some things, but as it’s easily automated it will help quickly catch many typical issues and thus has a natural role in the bigger Bayesian workflow.