The third most prevalent cancer nowadays is colorectal cancer. Colonoscopy is an important procedure in the stage of detection of polyps’ malignancy because it helps in early identification and establishes effective therapy. This paper explores specific deep-learning architectures for the binary classification of colorectal polyps and considers the evaluation of their premalignancy risk. The main scope is to create a custom-based deep learning architecture that classifies adenomatous, hyperplastic, and serrated polyps’ samples into benign and premalignant based on images from the colonoscopic dataset. Each image’s output is modified through masked autoencoders which enhance the classification performance of the proposed model, calledBionnica. From the four evaluated state-of-the-art deep learning models (ZF NET, VGG-16, AlexNet, and ResNet-50), our experiments showed that ResNet-50 and ZF NET are most accurate (above 84%), with ResNet-50 excelling at indicating patients with premalignant colorectal polyps (above 92%). ZF NET is the fastest at handling 700 images. Our proposed deep learning model,Bionnica, is more performant than ZF NET and provides an efficient classification of colorectal polyps given its simple structure. The advantage of our model comes from the custom enhancement interpretability with a rule-based layer that guides the learning process and supports medical personnel in their decisions.