Rice is one of the global most critical harvests, and a great many people eat it as a staple eating routine. Different rice plant diseases harm, spread, and drastically reduce crop yields. In extreme situations, they may result in no grain harvest at all, posing a severe threat to food security. In this paper, to amplify the recognition ability for rice leaf disease (RLD) classification, we proposed hierarchical transfer learning (HTL) methods incorporating ensemble models containing two-step. In the first step, an ensemble combining MobileNet and DenseNet was addressed to tackle the diseased leaf problem. Consequently, DenseNet and XceptionNet were fused to identify three RLDs. Here, we compare our models with state-of-the-art deep learning models such as ResNet, DenseNet, InceptionNet, Xception, MobileNet, and EfficientNet. Our framework at top-notch with 89 % and 91 % for accuracy. In future works, RLD segmentation is suggested to pinpoint the illness and quantify the afflicted region.