We present an analytical solution for the effect of thermorefractive noise considering finite-sized cylindrical test masses. For crystalline materials at low temperatures the effect of finite dimensions becomes important. The calculations are independently performed using the FluctuationDissipation-Theorem and Langevin's approach. Our results are applied to the input test mass of the current and future cryogenic gravitational wave detectors CLIO, LCGT, and ET and are compared to the respective standard quantum limit. For a substrate temperature of 10 K we find that the thermorefractive noise amplitude of the silicon input test mass in ET is only a factor of 2 below the standard quantum limit for frequencies above 500 Hz. Thus, thermorefractive noise of the input test mass could become a severe limitation if one uses techniques to beat the standard quantum limit like, e.g., squeezing. In contrast the effect of thermorefractive noise of the input test mass is negligible for CLIO and LCGT.