Simulating the real-time dynamics of lattice gauge theories, underlying the Standard Model of particle physics, is a notoriously difficult problem where quantum simulators can provide a practical advantage over classical approaches. In this work, we present a complete Rydberg-based architecture, co-designed to digitally simulate the dynamics of general gauge theories coupled to matter fields in a hardware-efficient manner. Ref. [1] showed how a qudit processor, where non-abelian gauge fields are locally encoded and time-evolved, considerably reduces the required simulation resources compared to standard qubit-based quantum computers. Here we integrate the latter with a recently introduced fermionic quantum processor [2], where fermionic statistics are accounted for at the hardware level, allowing us to construct quantum circuits that preserve the locality of the gauge-matter interactions. We exemplify the flexibility of such a fermion-qudit processor by focusing on two paradigmatic high-energy phenomena. First, we present a resource-efficient protocol to simulate the Abelian-Higgs model, where the dynamics of confinement and string breaking can be investigated. Then, we show how to prepare hadrons made up of fermionic matter constituents bound by non-abelian gauge fields, and show how to extract the corresponding hadronic tensor. In both cases, we estimate the required resources, showing how quantum devices can be used to calculate experimentally-relevant quantities in particle physics.