Non-Abelian gauge theories underlie our understanding of fundamental forces in nature, and developing tailored quantum hardware and algorithms to simulate them is an outstanding challenge in the rapidly evolving field of quantum simulation. Here we take an approach where gauge fields, discretized in spacetime, are represented by qudits and are time evolved in Trotter steps with multiqudit quantum gates. This maps naturally and hardware efficiently to an architecture based on Rydberg tweezer arrays, where long-lived internal atomic states represent qudits, and the required quantum gates are performed as holonomic operations supported by a Rydberg blockade mechanism. We illustrate our proposal for a minimal digitization of SU(2) gauge fields, demonstrating a significant reduction in circuit depth and gate errors in comparison to a traditional qubit-based approach, which puts simulations of non-Abelian gauge theories within reach of NISQ devices.