The persistence and productivity of forage grasses, important sources for feed production, are threatened by climate change-induced drought. Breeding programs are in search of new drought tolerant forage grass varieties, but those programs still rely on time-consuming and less consistent visual scoring by breeders. In this study, we evaluate whether Unmanned Aerial Vehicle (UAV) based remote sensing can complement or replace this visual breeder score. A field experiment was set up to test the drought tolerance of genotypes from three common forage types of two different species: Festuca arundinacea, diploid Lolium perenne and tetraploid Lolium perenne. Drought stress was imposed by using mobile rainout shelters. UAV flights with RGB and thermal sensors were conducted at five time points during the experiment. Visual-based indices from different colour spaces were selected that were closely correlated to the breeder score. Furthermore, several indices, in particular H and NDLab, from the HSV (Hue Saturation Value) and CIELab (Commission Internationale de l’éclairage) colour space, respectively, displayed a broad-sense heritability that was as high or higher than the visual breeder score, making these indices highly suited for high-throughput field phenotyping applications that can complement or even replace the breeder score. The thermal-based Crop Water Stress Index CWSI provided complementary information to visual-based indices, enabling the analysis of differences in ecophysiological mechanisms for coping with reduced water availability between species and ploidy levels. All species/types displayed variation in drought stress tolerance, which confirms that there is sufficient variation for selection within these groups of grasses. Our results confirmed the better drought tolerance potential of Festuca arundinacea, but also showed which Lolium perenne genotypes are more tolerant.