Uranium and thorium are toxic in different environments. The exploitation of uranium mines and associated mine drainage leaching towards streams, sediments, and soils cause relevant pollution. The U-mine areas present high concentrations of potentially toxic elements with several consequences to ecosystems and human health. Physicochemical and potentially toxic elements of mine dumps, stream sediments, and soils from the Canto Lagar uranium mine area (Central Portugal) were analyzed. Stream sediments, soils, and mine dumps show a large range in the concentration values of Fe, U, As, Cu, Zn, Pb, and Th, suggesting geological and mine contributions. Most of the selected potential toxic elements from sediments present a low to moderate contamination degree, except for As, W, and U, which vary between high and very high contamination index. The soils must not be used in agricultural or residential activities due to contamination in As and U. This abandoned mine represents an environmental risk due to the spatial mobility and dispersion of potentially toxic elements from the dumps to the sediments and soils, as well as by surface runoff and wind.