The scope of the present study is the estimation of the concentration of nitrates (NO3−) in groundwater using artificial neural networks (ANNs) based on easily measurable in situ data. For the purpose of the current study, two feedforward neural networks were developed to determine whether including land use variables would improve the model results. In the first network, easily measurable field data were used, i.e., pH, electrical conductivity, water temperature, air temperature, and aquifer level. This model achieved a fairly good simulation based on the root mean squared error (RMSE in mg/L) and the Nash–Sutcliffe Model Efficiency (NSE) indicators (RMSE = 26.18, NSE = 0.54). In the second model, the percentages of different land uses in a radius of 1000 m from each well was included in an attempt to obtain a better description of nitrate transport in the aquifer system. When these variables were used, the performance of the model increased significantly (RMSE = 15.95, NSE = 0.70). For the development of the models, data from chemical and physical analyses of groundwater samples from wells located in the Kopaidian Plain and the wider area of the Asopos River Basin, both in Greece, were used. The simulation that the models achieved indicates that they are a potentially useful tools for the estimation of groundwater contamination by nitrates and may therefore constitute a basis for the development of groundwater management plans.