The use of greywater heat exchangers (GHEs) is an effective way to reduce energy consumption for heating domestic water. However, the available characteristics of this type of device are often insufficient and consider only a few selected parameters of water and greywater, which results in the need to look for tools enabling the determination of the effectiveness of GHEs in various operating conditions with incomplete input data. The aim of this paper was to determine the usefulness of artificial neural networks (ANNs). For this purpose, comprehensive experimental tests were carried out on the effectiveness of the horizontal heat exchanger, taking into account a wide range of water and greywater flow rates and temperatures of these media, as well as the linear bottom slope of the unit, which allowed for the creation of a database of 32,175 results. Then, the feasibility of implementing the full research plan was assessed using ANNs. The analysis showed that the impact of the media temperatures on the heat exchanger effectiveness values obtained using ANNs is limited, which makes it possible to significantly reduce the number of necessary experiments. Adopting only three temperature values of at least one medium allowed the generation of ANN models with coefficient values R2 = 0.748–0.999 and RMSE = 0.077–1.872. In the case of the tested GHE, the slope and the flow rate of the mixed water are of key importance. However, even in the case of parameters of significant importance, it is possible to reduce the research plan without compromising the final results. Assuming five different values for each of the four input parameters (a total of 625 combinations) made it possible to generate an ANN model (R2 = 0.993 and RMSE = 0.311) with high generalization ability on the full research plan covering 32,175 cases. Therefore, the conducted analysis confirmed the usefulness of ANNs in assessing the effectiveness of GHEs in various operating conditions. The approach described in this paper is important for both environmental and economic reasons, as it allows for reducing the consumption of water and energy, which are necessary to carry out such scientific research.