This paper aims to evaluate whether an automatic analysis with deep learning convolutional neural networks techniques offer the ability to efficiently identify olive groves with different intensification patterns by using very high-resolution aerial orthophotographs. First, a sub-image crop classification was carried out. To standardize the size and increase the number of samples of the data training (DT), the crop images were divided into mini-crops (sub-images) using segmentation techniques, which used a different threshold and stride size to consider the mini-crop as suitable for the analysis. The four scenarios evaluated discriminated the sub-images efficiently (accuracies higher than 0.8), obtaining the largest sub-images (H = 120, W = 120) for the highest average accuracy (0.957). The super-intensive olive plantings were the easiest to classify for most of the sub-image sizes. Nevertheless, although traditional olive groves were discriminated accurately, too, the most difficult task was to distinguish between the intensive plantings and the traditional ones. A second phase of the proposed system was to predict the crop at farm-level based on the most frequent class detected in the sub-images of each crop. The results obtained at farm level were slightly lower than at the sub-images level, reaching the highest accuracy (0.826) with an intermediate size image (H = 80, W = 80). Thus, the convolutional neural networks proposed made it possible to automate the classification and discriminate accurately among traditional, intensive, and super-intensive planting systems.