Computer-aided diagnosis permits biopsy specimen analysis by creating quantitative images of brain diseases which enable the pathologists to examine the data properly. It has been observed from other image classification algorithms that the Extreme Learning Machine (ELM) demonstrates superior performance in terms of computational efforts. In this study, to classify the brain Magnetic Resonance Images as either normal or diseased, a hybridized Salp Swarm Algorithm-based ELM (ELM-SSA) is proposed. The SSA is employed to optimize the parameters associated with ELM model, whereas the Discrete Wavelet Transformation and Principal Component Analysis have been used for the feature extraction and reduction, respectively. The performance of the proposed “ELM-SSA” is evaluated through simulation study and compared with the standard classifiers such as Back-Propagation Neural Network, Functional Link Artificial Neural Network, and Radial Basis Function Network. All experimental validations have been carried out using two different brain disease datasets: Alzheimer’s and Hemorrhage. The simulation results demonstrate that the “ELM-SSA” is potentially superior to other hybrid methods in terms of ROC, AUC, and accuracy. To achieve better performance, reduce randomness, and overfitting, each algorithm has been run multiple times and a k-fold stratified cross-validation strategy has been used.