Milk and dairy products are among the most important food sectors in Georgia, and milk is considered one of the most essential foods in the human diet according to Georgian food culture. Kvemo Kartli is one of the major regions in Georgia for milk production. This region suffers from heavy metal contamination in soil and water because of the mining industry. This study was conducted to determine the concentrations of cadmium, lead, iron, zinc, copper, chromium, manganese, cobalt, nickel, selenium and molybdenum in milk and cheese and to evaluate whether the concentrations of these elements correspond to the permissible levels of toxic elements in milk and cheese for Georgia and the EU. In total, 195 milk samples and 25 cheese samples (16 from Imeruli cheese and nine from Sulguni cheese) were collected from nine different villages in the Kvemo Kartli region in Georgia: Chapala, Vanati, Bolnisi, Mtskneti, Sabereti, Ratevani, Khidiskuri, Kazreti, Kvemo Bolnisi. The determination of heavy metal in all samples was carried out by inductively coupled plasma-mass spectrometry. The research results show that the concentration of these elements in most milk samples is fairly constant for all villages and is less than the permissible levels, except for seven samples from the following villages: Kvemo Bolnisi, Bolnisi, Mitskineti and Ratawani, where the concentration of lead in the milk samples was higher than the permissible limits mentioned in the literature, ranging from 0.027 to 1003 mg L−1. As for copper, its concentration in milk in Sabereti and Vanati villages was above the permissible limits according to the EU limit, ranging from 0.42 to 1.28 mg L−1. For cheese samples, the concentration of cadmium, lead, copper, Co and Ni in the two types of cheese was less than the permissible limit according to the laws of Georgia. Finally, the heavy metal concentrations in Imeruli and Sulguni cheese for manganese (Mn), chromium (Cr), selenium (Se), molybdenum (Mo) zinc (Zn) and iron (Fe) were above the permissible limit. Thus, the study results showed that the consumption of milk does not pose a direct and serious threat to the health of consumers. As for the two types of cheese, future studies and continuous monitoring are necessary to assess the cheese content of trace elements and the risk of its consumption to the consumer.