Field portable X-ray fluorescence (pXRF) spectrometers are routinely used in mineral resources studies. To date, mineral resources studies have largely focussed on the application of pXRF to the exploration for deposits of base and precious metals. By contrast, studies using pXRF for the quantification of critical elements in geological materials are scarce since these elements are difficult to determine with energy-dispersive pXRF technology. This study explores the capability of pXRF spectrometers to detect and quantify critical elements (Ba, P, Nb, V, Co, REE, W, Bi, Hf, and Ta) in certified reference materials (CRMs). While precision of many critical elements is acceptable (<20% RSD), accuracy can be poor (>50% difference) when using pre-installed factory calibration software. Spectra collected during the pXRF measurements show that poor accuracy and false positives tend to be associated with spectral interferences. Distinct combinations of spectral interferences (line overlaps, Compton scattered peaks, and Si escape peaks) were observed in the different matrix types. Our results show that critical elements may be determined in common geological materials when pronounced peaks occur in the spectra and that matrix-match of standards and samples is essential. Hence, XRF spectra should be routinely reviewed to identify erroneous quantification due to spectral interferences.