Objectives: This study focuses on machine learning techniques to classify various stages of Alzheimer's Disease(AD). Methods: Absolutely, 1,997 PD weighted Resting State Functional MRI (rsFMRI) images were acquired from ADNI-3 dataset for the classification of AD. First, input rsFMRI images from the dataset were preprocessed and segmented. After segmentation, we have extracted multi variate features. Then, we have proposed Lasso with Graph Kernel Feature Selection (LGKFS) algorithm for selecting the best features. Finally, Radom Forest algorithm is applied to perform multi class classification for classifying all the stages of AD. Findings: In order to find the accuracy of this approach, cross validations were performed in the ADNI 3 dataset. We have measured the accuracy of RF classifier using three feature selection algorithms. The RF classifier with LASSO achieved 79.94% accuracy, 79.31% precision, 79.69% recall and 79.48% F1 score. The RF classifier with GK-FS achieved 76.52% accuracy, 84.0% precision, 79.19% recall and 79.77% F1 score respectively. By using our LGKFS algorithm, 90.8% accuracy, 82.4% precision ,81.6% recall and 81.6% F1 score was achieved by RF classifier which is higher than the existing feature selection techniques such as LASSO and GK-FS. Novelty: In this, a new hybrid feature selection algorithm namely LGKFS algorithm is introduced which combines two well-known feature selection algorithms Lasso Regression and GK-FS algorithm to improve classification accuracy.