Early detection of brain tumors is critical for effective treatment planning. Identifying tumors in their nascent stages can significantly enhance the chances of patient survival. While there are various types of brain tumors, each with unique characteristics and treatment protocols, tumors are often minuscule during their initial stages, making manual diagnosis challenging, time-consuming, and potentially ambiguous. Current techniques predominantly used in hospitals involve manual detection via MRI scans, which can be costly, error-prone, and time-intensive. An automated system for detecting brain tumors could be pivotal in identifying the disease in its earliest phases. This research applies several data augmentation techniques to enhance the dataset for diagnosis, including rotations of 90 and 180 degrees and inverting along vertical and horizontal axes. The CIELAB color space is employed for tumor image selection and ROI determination. Several deep learning models, such as DarkNet-53 and AlexNet, are applied to extract features from the fully connected layers, following the feature selection using entropy-coded Particle Swarm Optimization (PSO). The selected features are further processed through multiple SVM kernels for classification. This study furthers medical imaging with its automated approach to brain tumor detection, significantly minimizing the time and cost of a manual diagnosis. Our method heightens the possibilities of an earlier tumor identification, creating an avenue for more successful treatment planning and better overall patient outcomes.