In this study, the observer variability in a 10° achromatic color matching experiment using systematically different primaries under immersive, perceptually neutral (4673 K) and stable adapted viewing conditions has been investigated for 4 different sets of CIE color matching functions (CMFs). The matching reference was illuminated by a spectrally broadband light source at 70 cd/m² and 4613 K. Fifty-four observers varying in age, gender, ethnicity and eye color participated. The results indicate that certain wavelength regions are more sensitive to generate inter-observer variability, such as when a short wavelength primary (404 nm) was applied. Furthermore, of the primary sets investigated, that with peak-wavelengths (636-521-447 nm) close to the 'typical' RGBprimaries widely known and used in industry had the smallest inter-observer variability. In addition to the significant effect of primary set and CMF set, a significant effect of age on matching variability is also found, whereby the change of the blue primary resulted in a substantial difference in dispersion between younger and older observers, likely caused by age-induced yellowing of the lens. Furthermore, different primary sets tend to have a different impact on intra-observer variability than on inter-observer variability. Finally, by comparing the observer variability in the current work with previous studies, the CIE standard deviate observer is found to substantially underestimate the inter-observer variability, especially for the primary sets containing a 404 nm LED. Our observers also tend to have similar to smaller inter-variability than those reported in literature, possibly due to methodological differences.