SquirRL: Automating Attack Analysis on Blockchain Incentive Mechanisms with Deep Reinforcement Learning

Hou, Charlie; Zhou, Mingxun; Ji, Yan; Daian, Phil; Tramèr, Florian; Fanti, Giulia; Juels, Ari

doi:10.48550/arxiv.1912.01798

Cited by 11 publications

(21 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, a framework for identifying attacks against the incentive schemes of the blockchain protocols is proposed in [33]. In [16], proof of work blockchain protocols are modeled as stochastic games while in [47] a survey of game theoretic applications in the blockchain setting is presented.…”

Section: Other Related Workmentioning

confidence: 99%

Coalition-safe equilibria with virtual payoffs

Kiayias

Stouka

2021

Proceedings of the 3rd ACM Conference on Advances in Financial Technologies

View full text Add to dashboard Cite

show abstract

Section: Other Related Workmentioning

confidence: 99%

Coalition-safe equilibria with virtual payoffs

Kiayias

Stouka

2021

Proceedings of the 3rd ACM Conference on Advances in Financial Technologies

View full text Add to dashboard Cite

show abstract

“…Charlie et al proposed SquirRL which is a framework for using deep reinforcement learning to analyze attacks on blockchain incentive mechanisms. The revenue of SquirRL is greater than that of the Markov decision attackers when there are multiple selfish mining attackers [25].…”

Section: Related Workmentioning

confidence: 97%

“…They obtained the profitable threshold decreases. Charlie et al [25] proposed SquirRL framework for multiple attackers. This framework provides the optimal mining strategies based on the reinforcement learning.…”

Section: Introductionmentioning

confidence: 99%

Blockchain Mining with Multiple Selfish Miners

Bai¹,

Xu²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper studies a fundamental problem regarding the security of blockchain on how the existence of different numbers of misbehaving pools with different selfish mining strategies influences the profitability. Each selfish miner maintains a private chain and makes it public opportunistically for the purpose of acquiring more rewards incommensurate to his Hashrate. We establish a novel Markov chain model to characterize all the state transitions of public and private chains under basic selfish mining. The minimum requirement of Hashrate together with the minimum delay of being profitable is derived in close-form. The former reduces to 21.48% with the symmetric selfish miners, while the profitable threshold decreases with the increases of the number of attackers. The profitable delay increases with the decrease of the Hashrate of selfish miners, making the mining pools more cautious on performing selfish mining. We further investigate the profitability of selfish mining when one of the selfish miners performs the optimal attack based on Partially Observable Markov Decision Process (POMDP). An online search method is presented to compute the approximate optimal policy efficiently. Experimental results show that the optimal strategy significantly improves the revenue of the attacker.

show abstract

“…A new technique called SquirRL [11] analyzes blockchain protocols using deep reinforcement learning. It is based on the iterative MDP solution method of OSM, and uses deep-RL to find an approximately optimal policy.…”

Section: Related Workmentioning

confidence: 99%

Efficient MDP Analysis for Selfish-Mining in Blockchains

Zur

Eyal

Tamar

2020

Proceedings of the 2nd ACM Conference on Advances in Financial Technologies

View full text Add to dashboard Cite

A proof of work (PoW) blockchain protocol distributes rewards to its participants, called miners, according to their share of the total computational power. Sufficiently large miners can perform selfish mining-deviate from the protocol to gain more than their fair share. Such systems are thus secure if all miners are smaller than a threshold size so their best response is following the protocol. To find the threshold, one has to identify the optimal strategy for miners of different sizes, i.e., solve a Markov Decision Process (MDP). However, because of the PoW difficulty adjustment mechanism, the miners' utility is a non-linear ratio function. We therefore call this an Average Reward Ratio (ARR) MDP. Sapirshtein et al. were the first to solve ARR MDPs by solving a series of standard MDPs that converge to the ARR MDP solution. In this work, we present a novel technique for solving an ARR MDP by solving a single standard MDP. The crux of our approach is to augment the MDP such that it terminates randomly, within an expected number of rounds. We call this Probabilistic Termination Optimization (PTO), and the technique applies to any MDP whose utility is a ratio function. We bound the approximation error of PTO-it is inversely proportional to the expected number of rounds before termination, a parameter that we control. Empirically, PTO's complexity is an order of magnitude lower than the state of the art. PTO can be easily applied to different blockchains. We use it to tighten the bound on the threshold for selfish mining in Ethereum. CCS CONCEPTS • Security and privacy → Distributed systems security; • Theory of computation → Markov decision processes.

show abstract

SquirRL: Automating Attack Analysis on Blockchain Incentive Mechanisms with Deep Reinforcement Learning

Cited by 11 publications

References 54 publications

Coalition-safe equilibria with virtual payoffs

Coalition-safe equilibria with virtual payoffs

Blockchain Mining with Multiple Selfish Miners

Efficient MDP Analysis for Selfish-Mining in Blockchains

Contact Info

Product

Resources

About