Cancer is one of the foremost reasons of death worldwide, with nearly 10 million deaths noted in 2020. Globally oral cancer has the sixth rank when compared to other cancers. It is lethal because most cases are noticed at advanced stages, which can be prevented if screened for or treated early, leading to a significant decrease in the mortality rate due to oral cancer. Thus, it becomes necessary to detect oral cancer at its pre-cancerous stages. In this work, a method for classifying oral cavity lesions into benign and malignant for binary classification and their pre-cancerous stages for multi-class classification exploring five different color spaces extracting color and texture features classified using a machine learning technique Light Gradient Boosting Machine (LightGBM) is proposed. The overall performance is satisfactory, outperforming the state-of-art methods in this task, with a testing accuracy of 99.25%, precision of 99.18%, recall of 99.31%, f1-score of 99.24% and specificity of 99.31% for the binary classification, and testing accuracy of 98.88%, precision of 98.86%, recall of 97.92%, f1-score of 98.38% and specificity of 99.03% for multi-class classification. The proposed method used hand-crafted features and a machine-learning classifier, making it less time-consuming and using bare minimum resources.INDEX TERMS Binary and multi-class classification, color spaces, early detection, feature importance, oral cancer, white light images