The Kingdom of Saudi Arabia is considered to be one of the world leaders in olive production accounting for about 6% of the global olive production. Given the fact that 94% of the olive groves are mainly rain-fed using traditional methods of production, the annual olive production is witnessing a noticeable fluctuation which is worse due to infectious diseases and climate change. Thus, early and effective detection of plant diseases is both required and urgent. Most farmers use traditional methods, for example, visual inspection or laboratory examination, to identify plant diseases. Currently, deep learning (DL) techniques have been shown to be useful methods for diagnosing olive leaf diseases and many other fields. In this work, we use a deep feature concatenation (DFC) mechanism to combine features extracted from input images using the two modern pretrained CNN models, i.e., ResNet50 and MobileNet. Hence, we propose MobiRes-Net: A neural network that is a concatenation of the ResNet50 and MobileNet models for overall improvement of prediction capability. To build the dataset used in the study, 5400 olive leaf images were collected from an olive grove using a remote-controlled agricultural unmanned aerial vehicle (UAV) equipped with a camera. The overall performance of the MobiRes-Net model achieved a classification accuracy of 97.08% which showed its superiority over ResNet50 and MobileNet that achieved classification accuracies of 94.86% and 95.63%, respectively.