Background: There is variable reporting on the benefits of a 200 μg/d selenium supplementation towards reducing prostate cancer impacts. The current analysis is to understand whether stratified groups receive supplementation benefits on prostate health. Methods: 572 men were supplemented with 200 µg/d selenium as selinized yeast for six months, and 481 completed the protocol. Selenium and prostate-specific antigen (PSA) levels were measured in serum at pre- and post-supplementation. Changes in selenium and PSA levels subsequent to supplementation were assessed with and without demographic, lifestyle, genetic and dietary stratifications. Results: The post-supplementation selenium (p = 0.002) and the gain in selenium (p < 0.0001) by supplementation were significantly dependent on the baseline selenium level. Overall, there was no significant correlation between changes in PSA and changes in selenium levels by supplementation. However, stratified analyses showed a significant inverse correlation between changes in PSA and changes in selenium in men below the median age (p = 0.048), never-smokers (p = 0.031), men carrying the GPX1 rs1050450 T allele (CT, p = 0.022 and TT, p = 0.011), dietary intakes above the recommended daily intake (RDI) for zinc (p < 0.05), and below the RDI for vitamin B12 (p < 0.001). Conclusions: The current analysis shows the influence of life factors on prostate health benefits of supplemental selenium.