Background: The release of environmental pollutants such as heavy metals due to mining activities has received worldwide condemnation. Therefore, understanding the contamination of both surface and groundwater in Edege-Mbeki mining district is very important for the health and wellbeing of its people. Hence, the purpose of this study was to determine the physicochemical parameters of water samples and the level of heavy metals in Edege-Mbeki Columbite/Tantalite (COLTAN) mining site, North Central Nigeria. Results: A total of 15 water samples were collected and analyzed in triplicate. The concentrations of five heavy metals (Ni, Mn, Pb, Cd, and Fe) in surface and groundwater samples were measured using atomic absorption spectrophotometer (AA-6800 Shimadzu). The physicochemical properties of the water samples were also examined. The results of the water quality assessments show that the water samples were contaminated with alarming levels of Pb, Cd and Ni. Mean concentrations were Ni (0.008 mg/L), Mn (0.087 mg/L), Pb (0.324 mg/L), Cd (0.029 mg/L), and Fe (0.839 mg/L) in Edege-Mbeki groundwater, while the surface water (mining ponds) had Ni (0.102 mg/L), Mn (0.099 mg/L), Pb (0.358 mg/L), Cd (0.025 mg/L), and Fe (6.99 mg/L). The groundwater samples in Mararaban-Edege which served as the control site had mean concentration levels of Ni (0.032 mg/L), Mn (0.096 mg/L), Pb (0.122 mg/ L), Cd (0.039 mg/L), and Fe (0.608 mg/L). The range of pH was found to be from 5.6 ± 0.02 to 6.5 ± 0.12, turbidity was between 39.27 ± 0.29 and 2271 ± 0.00 NTU, and conductivity ranged from 39.27 ± 0.29 to 757.33 ± 5.69 μS/cm, TDS from 25.0 ± 2.3 to 450.67 ± 6.03 mg/L while dissolved oxygen (DO) values from 3.1 ± 0.3 to 3.5 ± 0.3 mg/L. Conclusion: This study confirmed that surface and ground water samples in Edege-Mbeki and Mararaban-Edege are not fit for human consumption and other domestic use. Hence, government and community leaders must provide alternative means of water for the people of these communities.