Groundwater is an essential source of drinking and irrigation water. However, elevated Fe and Mn concentrations in groundwater have been found in recent decades, which can adversely affect human health and decrease crop quality and yields. The roles of hydrogeochemical changes and groundwater pollution (exogenous reductive material inputs) in this have not been studied adequately. We determined the distribution of Fe and Mn concentrations in groundwater in the Songnen Plain, northeast China, which is known for elevated Fe and Mn concentrations, and investigated the factors and mechanisms involved in causing the elevated concentrations. Chemical and statistical analyses indicated that the Fe and Mn concentrations in groundwater significantly correlated with climate parameters (precipitation and temperature), surface features (altitude, distance from a river, soil type, soil texture, and land use type) and hydrogeochemical characteristics (chemical oxygen demand and NH4+, NO3−, and P concentrations). In particular, the Fe and Mn concentrations in groundwater are higher in areas containing paddy fields and water bodies than other land use type areas. Areas with groundwater containing ultra-high Fe and Mn concentrations have almost all of the favorable factors. The main reasons for the elevated Fe and Mn concentrations in groundwater in the study area are the Fe/Mn mineral-rich strata and soil with abundant organic matter acting as sources of Fe and Mn to the groundwater and the reductive environment in the lower terrain and areas containing water bodies favoring Fe and Mn dissolution in the groundwater. Inputs of pollutants from agricultural activities have caused the Fe and Mn concentrations in groundwater to increase. Future studies should be performed to study interactions between pollutants from agricultural activities and Fe and Mn in groundwater and develop environmental management strategies for preventing future increases in Fe and Mn concentrations and promoting sustainable development of agriculture.