Fly ash, as a supplemental pozzolanic material, reduces concrete’s adverse environmental footprint by decreasing the emission of carbon dioxide (CO2) during the cement manufacturing process. Fly ash, which is a waste material, can enhance both the mechanical characteristics and durability of concrete, and has the capability to play an important role in sustainable design. Considering the widespread interest in applying Fly ash, and despite research studies, the level of replacement is still unclear. In this paper, a novel method using artificial neural networks (ANN) is presented to predict concrete’s mechanical characteristics by adding Fly ash. In this regard, a host of available experimental data, such as the properties of Fly ash, along with concrete additives, was fed into an ANN model. Concrete samples’ tensile and compressive strengths, in addition to their modulus of elasticity, were defined as outputs. It was observed that the predicted outcomes agreed well with the experimental results. To further enhance the research outcomes, simple but practical equations are presented to assess the effect of using Fly ash on concrete’s mechanical characteristics.