Computing nonsurjective primes associated to Galois representations of genus 2 curves

Abstract

For a genus 2 2 curve C C over Q \mathbb {Q} whose Jacobian A A admits only trivial geometric endomorphisms, Serre’s open image theorem for abelian surfaces asserts that there are only finitely many primes ℓ \ell for which the Galois action on ℓ \ell -torsion points of A A is not maximal. Building on work of Dieulefait, we give a practical algorithm to compute this finite set. The key inputs are Mitchell’s classification of maximal subgroups of P S p 4 ( F ℓ ) PSp_4(\mathbb {F}_\ell ) , sampling of the characteristic polynomials of Frobenius, and the Khare–Wintenberger modularity theorem. The algorithm has been submitted for integration into Sage, executed on all of the genus 2 2 curves with trivial endomorphism ring in the LMFDB, and the results incorporated into the homepage of each such curve.