Adaptive utility and trial aversion

Abstract

Decision making with adaptive utility provides a generalisation to classical Bayesian decision theory, allowing the creation of a normative theory for decision selection when preferences are initially uncertain. In this paper we address some of the foundational issues of adaptive utility as seen from the perspective of a Bayesian statistician. The implications that such a generalisation has upon the traditional utility concepts of value of information and risk aversion are also explored, with a new concept of trial aversion introduced that is similar to risk aversion, but which concerns a decision maker's aversion to selecting decisions with high uncertainty over resulting utility.

Nonparametric predictive utility inference

“…In selecting one over the alternative, a DM will indicate a level of trial aversion, with EP indicating extreme trial aversion and a preference for the known, whilst EO indicates extreme trial seeking behaviour and a preference for the unknown. The theory of trial aversion was introduced by [26] and is an analogous, but orthogonal, concept to the usual risk aversion of [30] and [2], but only arising in theories of decision making under uncertain preferences.…”

“…In contrast, the theory of adaptive utility, as introduced by [13] and further developed by [25] and [24,26], generalises classical Bayesian decision theory, suggesting a methodology for decision selection even when the DM is unable to fully specify her preferences. In this setting the DM is permitted to be uncertain over her true preferences, with the appropriate utility function over decision outcomes only being known up to the value of some uncertain parameter µ.…”

“…This is a general problem of modelling uncertain utility, and to the best of our knowledge, there is not yet any practical solution that does not rely on similarly unreasonable assumptions. Previous possibilities have been to include scaling subject to commensurability assumptions in which the same outcome is always deemed most preferable, e.g., in [7] and [26], but this does not allow uncertainty as to which outcome is most preferred. As such, we here suggest the interpretation that the utilities of experienced outcomes have been placed on the [0, 1] scale by considering 'hypothetical' best and worst possible outcomes that could exist within the exchangeable collection and by assigning those hypothetical outcomes to have utilities 1 and 0, respectively.…”

“…Furthermore, as there is no uncertainty in the utility of option f 5 , both the lower and upper expected utility bounds for the EP and EO choice rules also coincide. However, under the EO choice rule, the expected value of information gained following the initial experimentation of an unfamiliar outcome will increase as the length of the planning horizon is also increased, and this is because if the unfamiliar outcome were found to have greater utility, then it would be re-selected in every future epoch, see for example [31] and [26]. Hence, if the planning horizon of the strategy was extended beyond three decision epochs, then there would exist a horizon for which the EO choice rule would suggest initial experimentation.…”

“…Indeed, a DM may consider it inappropriate to assign a particular and fixed utility value for any outcome that is novel or unfamiliar, choosing instead to only do so after direct experience or exposure. Such cases of utility uncertainty motivate so-called adaptive utility theory, e.g., [13], [25], and [24,26], which generalizes the traditional utility concept by only requiring the utility function be known up to the value of some uncertain utility parameter. The principal idea of adaptive utility is then to treat the uncertain utility parameter in the same manner that unknown random quantities are typically treated in standard Bayesian statistical inference, i.e., they are subjected to a parametric learning model in accordance with Bayes' Theorem.…”