Non-Abelian geometric phases have been proposed as an essential ingredient in logical gate implementation—their geometric nature guarantees their invariance under reparameterizations of the associated cyclic path in parameter space. However, they are still dependent on deformations of that path due to noise. The first question that we tackle in this work is how to quantify in a meaningful way this effect of noise, focusing, for concreteness, on the nuclear quadrupole resonance Hamiltonian—other systems of this nature can clearly be treated analogously. We consider a precessing magnetic field that drives adiabatically a degenerate doublet and is subjected to noise, the effects of which on the Wilczek–Zee holonomy are computed analytically. A critical review of previous related works reveals a series of assumptions, such as sudden jumps in the field, or the presence of white noise, that might violate adiabaticity. We propose a state-independent measure of the effect and then consider sinusoidal noise in the field of random amplitude and phase. We find that all integer noise frequencies m ≠ 2 behave similarly in a manner reminiscent of the Abelian case, but that noise of frequency m = 2 has a very different and, at the same time, very pronounced effect, which might well affect robustness estimations.