The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations Using an Interactive Shiny App

Abstract

The current paper highlights a new, interactive Shiny App that can be used to aid in understanding and teaching the important task of conducting a prior sensitivity analysis when implementing Bayesian estimation methods. In this paper, we discuss the importance of examining prior distributions through a sensitivity analysis. We argue that conducting a prior sensitivity analysis is equally important when so-called diffuse priors are implemented as it is with subjective priors. As a proof of concept, we conducted a small simulation study, which illustrates the impact of priors on final model estimates. The findings from the simulation study highlight the importance of conducting a sensitivity analysis of priors. This concept is further extended through an interactive Shiny App that we developed. The Shiny App allows users to explore the impact of various forms of priors using empirical data. We introduce this Shiny App and thoroughly detail an example using a simple multiple regression model that users at all levels can understand. In this paper, we highlight how to determine the different settings for a prior sensitivity analysis, how to visually and statistically compare results obtained in the sensitivity analysis, and how to display findings and write up disparate results obtained across the sensitivity analysis. The goal is that novice users can follow the process outlined here and work within the interactive Shiny App to gain a deeper understanding of the role of prior distributions and the importance of a sensitivity analysis when implementing Bayesian methods. The intended audience is broad (e.g., undergraduate or graduate students, faculty, and other researchers) and can include those with limited exposure to Bayesian methods or the specific model presented here.

Probabilistic modelling of developmental neurotoxicity based on a simplified adverse outcome pathway network

“…A drawback of Bayesian models is often considered to be the subjectivity of priors. Even though the selection of priors is always debatable, ultimately, the role of priors is to improve the predictions, e.g., the process of data shrinkage towards a prior group-mean to represent the common group distribution [62] . Besides, the AOPs themselves are subjective representations of adverse effects, and hence, there are other subjective elements in the model building process, as argued by van de Schoot et al [21] .…”

Performance of bovine genital campylobacteriosis diagnostic tests in bulls from Uruguay: a Bayesian latent class model approach

“…To determine the impact of the prior on the posterior densities, two additional models were ran using weakly informative priors on the prevalence or for the tests Se and Sp (Depaoli et al 2020 ; Natesan Batley and Hedges 2021 ; Paradis et al 2012 ). In model 2, we used relaxed estimates for the prevalence in the two populations.…”

A Review of Bayesian Hypothesis Testing and Its Practical Implementations

“…[ 49 ] believes that using an informative prior expressing a psychological theory and evaluating models using prior sensitivity measures can serve to advance knowledge. Finally, sensitivity analysis is accessible through an interactive Shiny Application developed by the authors in [ 50 ]. The software is designed to help user understand how to assess the substantive impact of prior selection in an interactive way.…”