Experiments on choice under risk typically involve multiple decisions by individual subjects. The choice of mechanism for selecting decision(s) for payoff is an essential design feature unless subjects isolate each one of the multiple decisions. We report treatments with different payoff mechanisms but the same decision tasks. The data show large differences across mechanisms in subjects' revealed risk preferences, a clear violation of isolation. We illustrate the importance of these mechanism effects by identifying their implications for classical tests of theories of decision under risk. We discuss theoretical properties of commonly used mechanisms, and new mechanisms introduced herein, in order to clarify which mechanisms are theoretically incentive compatible for which theories. We identify behavioral properties of some mechanisms that can introduce bias in elicited risk preferences-from cross-task contaminationeven when the mechanism used is theoretically incentive compatible. We explain that selection of a payoff mechanism is an important component of experimental design in many topic areas including social preferences, public goods, bargaining, and choice under uncertainty and ambiguity as well as experiments on decisions under risk.