The influence of different irrigation regimes on olive fruit morphological parameters and on the quantity and quality (marketable indices, phenolic content, fatty acid composition, and sensory profile) of virgin olive oil (VOO) obtained from the Croatian cultivar Oblica, grown on an extremely rocky and dry reclaimed karst soil, was studied over three years. Four treatments were applied: rain-fed and three treatments calculated as 50%, 75%, and 100% of the crop’s irrigation requirement (Irr). Principal component analysis separated growing seasons (GS) that differed in precipitation. In the 2016 season, which had a low number of fruits per kilogram and provided a higher amount of balanced VOO with medium to intense bitterness and pungency (rain-fed treatment), the oil yield increased by irrigation (Irr 75 and Irr 100) up to 18%, while unchanged phenolics, bitterness, and pungency were observed for the VOOs obtained. In the drier GS (2017), which under rain-fed conditions had high fruit per kg, smallest fruit sizes, and lowest oil yield, and in which the VOOs had high phenolic content and intense sensory taste attributes, fruit weight, fruit sizes, and oil yield increased by 35% in all irrigation treatments, while phenols, bitterness, and pungency decreased, balancing the sensory profile of the VOOs. The results obtained here led us to conclude that the irrigation of young olives resulted in a positive effect, with the indication that an abundant water supply is more effective in drought conditions.