Generalizations of the classic Bell inequality to higher dimensional quantum systems known as qudits are reputed to exhibit a higher degree of robustness to noise, but such claims are based on one particular noise model. We analyze the violation of the Collins-Gisin-Linden-Massar-Popescu inequality subject to more realistic noise sources and their scaling with dimension. This analysis is inspired by potential Bell inequality experiments with superconducting resonator-based qudits. We find that the robustness of the inequality to noise generally decreases with increasing qudit dimension.