We propose an action of a certain motivic cohomology group on the coherent cohomology of Hilbert modular varieties, extending conjectures of Venkatesh, Prasanna, and Harris. The action is described in two ways: on cohomology modulo $p$ and over ${\mathbb {C}}$, and we conjecture that they both lift to an action on cohomology with integral coefficients. The conjecture is supported by theoretical evidence based on Stark’s conjecture on special values of Artin $L$-functions and by numerical evidence in base change cases.