Buildings expand and contract in response to their environment, which results in cracks in the structure. This can pose a serious threat to the people who use it, and these movements are frequently too small to be observed, and thus go unnoticed. Cracks can be caused by a variety of factors, including defects in the construction process, ground movement, foundation failure, and decay of the building fabric. If a structure is unable to accommodate this movement, cracking is likely to occur, posing a serious risk to the building's structural integrity. Only after cracks are identified can they be treated, and existing manual methods of sketching the crack patterns are highly subjective to the person performing the analysis, are frequently constrained by high costs, equipment and tool availability, and are extremely time consuming. In this paper, 40,000 images divided into two and categorized into positive and negative cracks are used as input and the presence of cracks is detected using a deep learning technique. The following crack types are included in the experimentation: hairline, stepped, vertical, and horizontal. In comparison to conventional image processing and other deep learning-based techniques, the proposed Convolutional Neural Network (CNN) achieves significantly higher accuracy than the Recurrent Neural Network (RNN). This paper’s objective is to create a model which can detect the cracks through deep learning methodology, and this will be the innovative region in crack detection using neural net framework.