Medical image processing relies heavily on the diagnosis of brain tumor images. It aids doctors in determining the correct diagnosis and management. One of the primary imaging methods for studying brain tissue is MR imaging. In recent years, deep learning techniques have shown significant potential in image processing. However, the modest quantity of medical images is a restriction of the classification of medical images. As a result of this restriction, fewer medical photos are available. Fine-tuned ResNet-101 (FR-101) is proposed to classify the brain tumor images to counteract this issue. Weiner filter is used to de-noise the acquired raw MR images, and the adaptive histogram equalization technique is used to improve contrast. A stacked autoencoder is utilized in the segmentation procedure to separate the tumor from healthy brain parts from the preprocessed data. The marker-based watershed technique is used to identify the tumor location and structure in the segmented data. The recommended approach is then used in the classification stage. To obtain the highest level of accuracy for our research, accuracy, precision, f1-score, recall, and mean absolute error are the measures of success are studied as well as a comparison of the suggested approach with a few other existing methods.