Due to the fact of water resource deterioration from human activities and increased demand over the last few decades, optimization of management practices and policies is required, for which more reliable data are necessary. Cost and time are always of importance; therefore, methods that can provide low-cost data in a short period of time have been developed. In this study, the ability of an artificial neural network (ANN) and a multiple linear regression (MLR) model to predict the electrical conductivity of groundwater samples in the GallikosRiver basin, northern Greece, was examined. A total of 233 samples were collected over the years 2004–2005 from 89 sampling points. Descriptive statistics, Pearson correlation matrix, and factor analysis were applied to select the inputs of the water quality parameters. Input data to the ANN and MLR were Ca, Mg, Na, and Cl. The best results regarding the ANN were provided by a model that included one hidden layer of three neurons. The mean absolute percentage error, modeling efficiency, and root mean square error were used to evaluate the performances of the methods and to compare the prediction capabilities of the ANN and MLR. We concluded that the ANN and MLR models were valid and had similar accuracy (using the same inputs) with a large number of samples, but in the case of a smaller data set, the MLR showed a better performance.