The aim of the present study was to measure lanthanum (La), cerium (Ce), europium (Eu), and gadolinium (Gd) concentrations in human semen and correlate the results with sperm quality. The median semen content of La was 19.5 µg kg−1 dry weight (dw) (range 2.27–269), of Ce was 41.9 µg kg−1 dw (range 4.52 to 167), of Eu was 0.68 µg kg−1 dw (range 0.06–1.95), of Gd was 3.19 µg kg−1 dw (range 0.38–12.0), and of calcium (Ca) was 4063 mg kg−1 dw (range 484–17,191). Concentrations of La, Ce, Eu, Gd, and Ca were significantly lower in nondrinkers’ semen than in semen from drinkers. Significant differences were detected between La, Ce, Eu, Gd, and Ca concentrations in semen from nondrinkers and moderate drinkers. Concentrations of La, Ce, and Gd in semen of short-term smokers were significantly lower than those in extremely long-term smokers. Significant differences were also detected between La concentration in semen from a group of short-term smokers and that of a group of long-term smokers. Positive correlations were found between La, Ce, Eu, Gd, and Ca concentrations in semen. La, Ce, Gd, and Ca concentrations in semen were positively associated with progressive motility and percentage of normal spermatozoa. Positive correlations were found between Ca and sperm concentration. Concentrations of La, Ce, and Gd were negatively associated with sperm concentration, whilst Ca concentration was negatively associated with volume of ejaculate. At the examined level, La, Ce, Eu, and Gd did not affect sperm quality, whereas alcohol consumption and smoking might have increased the level of rare earth elements in semen.