Achieving high-fidelity entangling operations between qubits consistently is essential for the performance of multi-qubit systems. Solid-state platforms are particularly exposed to errors arising from materials-induced variability between qubits, which leads to performance inconsistencies. Here we study the errors in a spin qubit processor, tying them to their physical origins. We use this knowledge to demonstrate consistent and repeatable operation with above 99% fidelity of two-qubit gates in the technologically important silicon metal-oxide-semiconductor quantum dot platform. Analysis of the physical errors and fidelities in multiple devices over extended periods allows us to ensure that we capture the variation and the most common error types. Physical error sources include the slow nuclear and electrical noise on single qubits and contextual noise that depends on the applied control sequence. Furthermore, we investigate the impact of qubit design, feedback systems and robust gate design to inform the design of future scalable, high-fidelity control strategies. Our results highlight both the capabilities and challenges for the scaling-up of silicon spin-based qubits into full-scale quantum processors.