Pascal’s mugging

“…cases in which expected value calculations are dominated by tiny probabilities of extreme scenarios (for example, Pascal's wager [Pascal 1669], Pascal's mugging [Bostrom 2009], or the St. Petersburg game [Bernoulli 1738]). Merely noting this similarity serves at best to categorize the solipsistic swamping problem, not to solve it.…”

Section: Escape Routesmentioning

confidence: 99%

Average Utilitarianism Implies Solipsistic Egoism

Tarsney¹

2021

Australasian Journal of Philosophy

View full text Add to dashboard Cite

Average utilitarianism and several related axiologies, when paired with the standard expectational theory of decision-making under risk and with reasonable empirical credences, can find their practical prescriptions overwhelmingly determined by the minuscule probability that the agent assigns to solipsism-that is, to the hypothesis that there is only one welfare subject in the world, namely, herself. This either (i) constitutes a reductio of these axiologies, (ii) suggests that they require bespoke decision theories, or (iii) furnishes an unexpected argument for ethical egoism.

show abstract

“…This is Pascal's Wager, which has been criticized on many grounds, including conceptual difficulties with infinite utilities. To address this, Yudkowsky (2007) and Bostrom (2009) have proposed Pascal's Mugging as a finite version capturing much of Pascal's initial line of reasoning. Pascal's Mugging is said to occur when you threaten to impose an astronomical punishment on someone unless they give you resources, and your victim estimates that the probability of you being able to impose the punishment is extremely close to zero but still higher enough so that the expected cost of the punishment is greater than the resources you demand.…”

Section: Pascal's Muggingmentioning

confidence: 99%

An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis

2020

View full text Add to dashboard Cite

An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the strong orthogonality thesis since the modifications would be influenced by the AGI’s intelligence. AGIs in highly competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory. Nothing in our analysis weakens arguments concerning the risks of AGI.

show abstract

Pascal’s mugging

Cited by 7 publications

References 0 publications

1% Skepticism

1% Skepticism

Average Utilitarianism Implies Solipsistic Egoism

An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis

Contact Info

Product

Resources

About