Groundwater holds an important role in the water supply in Linyi city, China. Investigating the hydrochemical characteristics of groundwater, and revealing the factors governing groundwater geochemistry, is a primary step for ensuring the safe and rational exploitation of groundwater resources. This study used a self-organizing map (SOM) and multivariate statistical methods to assess groundwater quality in the urban area of Linyi city. Based on the hydrochemical dataset consisting of nine parameters (i.e., pH, Ca2+, Mg2+, Na+, K+, HCO3−, Cl−, SO42−, and NO3−) from 89 groundwater samples, the SOM was first applied to obtain the weight vectors of the output nodes. Hierarchical cluster analysis (HCA) was used for organizing the nodes into four clusters. The node cluster indices were then remapped to the groundwater samples according to the winner node for each sample. The hydrochemical characteristics and factors controlling the groundwater geochemistry of the four clusters were analyzed using principal component analysis (PCA) and graphical methods including Piper and Gibbs diagrams, as well as binary plots of the major ions in groundwater. Results indicated that groundwater geochemistry in this area is primarily governed by water–rock interactions, such as the dissolution of halite, calcite, and gypsum, along with the influence of municipal sewage and the degradation of organic matter. This study demonstrates that the integration of an SOM and multivariate statistical methods improves the understanding of groundwater geochemistry and hydrochemical evolution in complex groundwater flow systems impacted by utilization.