Yield prediction is a crucial activity in scheduling agronomic operations and in informing the management and financial decisions of a wide range of stakeholders of the organic durum wheat supply chain. This research aims to develop a yield forecasting system by combining vegetation index time-series data from Sentinel-2 L2A time-series data, field-measured yields, and deep learning techniques. Remotely sensed data over a season could be, in general, noisy and characterized by a variable density due to weather conditions. This problem was mitigated using Functional Principal Component Analysis (FPCA). We obtained a functional representation of acquired data, and starting from this, we tried to apply deep learning to predict the crop yield. We used a Convolutional Neural Network (CNN) approach, starting from images that embed temporal and spectral dimensions. This representation does not require one to a priori select a vegetation index that, typically, is task-dependent. The results have been also compared with classical approaches as Partial Least Squares (PLS) on the main reference vegetation indexes such as the Normalized Difference Vegetation Index (NDVI) and Normalized Difference Red Edge index (NDRE), considering both in-season and end-season scenarios. The obtained results show that the image-based representation of multi-spectral time series could be an effective method to estimate the yield, also, in the middle stage of cropping with R2 values greater than 0.83. The developed model could be used to estimate yield the neighbor fields characterized by similar setups in terms of the crop, variety, soil, and, of course, management.