Plants are extensively dispersed in nature, and they are essential for the survival and growth of the entire living creatures on the planet. Leaf classification and identification are critical in botany for recognizing new or endangered tree species. Leaf classification must be done quickly and accurately to sustain agricultural products. Traditional methods used manual classification of plant species. However, this method is costly, time‐consuming, and in some circumstances, impracticable. In the domains of image classification, target identification, and other fields, the new generation of Convolutional Neural Networks (CNNs) has shown outstanding outcomes. However, there exist some practical difficulties in executing these artificial neural networks. Owing to its high complexity and colossal running time, and also insufficient efficiency, the performance of classification is affected. An inappropriate count of neurons at each hidden layer may lead to an overfitting or underfitting problem. Thus, the core concept of this task is concerned with a novel technique for the growth of plant leaf classification for untrained data by hybridized classifiers. The core concept of this task is to find untrained data. The leaf images are collected from online sources. The images are further fed to the preprocessing phase. Pattern extraction is done on the preprocessed image using Weighted Weber Local Pattern. This preprocessed image is used for the classification of untrained data using hybridized deep‐structured architectures. Here, the combination of CNN with Recurrent Neural Network (RNN) and CNN with Support Vector Machine (SVM) is utilized for classification. Averaging the classification score of both hybrid techniques with parameter and threshold being optimized by the Best Searchable Modified Harris Hawks Optimization (BM‐HHO) produces the best classification outcome for untrained data. Here, the corresponding leaf type is classified if the input is given as the trained leaf images. Simultaneously, if the other types of data, like, palm tree leaf images, are given as input, and then explicitly shows the output as another type of leaves (or) untrained data. The experimental outcomes on the benchmark data set containing leaf images reveal that the introduced method achieved a higher accuracy than the conventional techniques. The accuracy of BM‐HHO‐CRNN + CSVM at the 65th learning rate is 3.43%, 0.84%, 1.80%, and 1.05% superior to HHO‐CRNN + CSVM, SS‐WOA‐CRNN + CSVM, WOA‐CRNN + CSVM, and SSO‐CRNN + CSVM.