Cadmium (Cd) contamination is a growing concern, as exposure to the metal has been shown to inhibit plant growth and development. However, soil Cd pollution in China is typically mild, and thus its concentration often does not impede plant growth. On the other hand, it is unknown if increased plant growth impacts Cd uptake, movement, and accumulation. Here, we analyzed the relationship between Cd accumulation in 31 tomato cultivars and the impact on specific growth parameters in mild Cd contamination. The results showed that there are variations in the Cd distribution among the 31 tomato cultivars studied. There were higher Cd concentrations in shoots of the cultivar ‘SV3557’, whereas root Cd concentrations were the lowest. The roots of the cultivar ‘HF11’ recorded the lowest Cd content but had higher Cd content in the shoots. The Cd concentration in roots and shoots was not related to root length, plant height, and root weight. However, Cd accumulation in the shoots was markedly promoted by root length and plant height, and Cd accumulation in the roots was promoted by root weight. Subsequently, we imposed Cd on four selected tomato cultivars to ascertain their accumulation in the shoot tissues. The results revealed that, among the four tomato cultivars, Cd was highly accumulated in the leaves, followed by the stems, and the fruits (leaf > stem > fruit). When identifying significant loci associated with Cd accumulation in tomato plants, it is crucial to find a suitable indicator to assess the plant’s ability to accumulate Cd. Thus, Cd concentration in shoots can be used as a reliable proxy for evaluating tomato plants’ capacity for Cd accumulation. This study serves as a valuable reference in guiding the selection of such an index.