In the field of agriculture, identifying and classifying Oryza Sativa diseases is always a hot research topic. Almost in all the states of India, Oryza Sativa (rice) is cultivated as a necessary food crop. However, rice plants are highly caused due to diseases that drastically affect the agricultural sectors. Researchers are searching for solutions for reliable and exact detection approaches for crop leaf diseases. A Oryza Sativa plant leaf disease classification system that is modelled relying on the concept of deep learning-driven Least Square Support Vector Machine is investigated in the present study. This work focus on the Oryza Sativa plant leaf diseases like Brown Leaf spot(BS), Bacterial leaf blight (BLB), Leaf smut(LS), Tungro(LT), and Leaf Blast(LB). To create a contrast-enhanced image dataset, the input Oryza Sativa plant disease image dataset is resized as well as histogram equalization and the Kuwahara filter are applied. The diseased part is segmented using spatial fuzzy clustering (SFC). Then, a multi-model LeNet5 is used in extracting features, and the least square support vector machine(LSSVM) is employed to classify the Oryza Sativa plant leaf diseases images. Further, to improve the accuracy of the disease classification Weighted Sparrow Search Optimization (WSSO) is utilized. The proposed multi-model LSSVM-WSSO outperformed the existing models on the two benchmark datasets when evaluated on the terms of accuracy, precision, recall, and F1-measure values.