Olive (Olea europeae L.) salinity stress induces responses at morphological, physiological and molecular levels, affecting plant productivity. Four olive cultivars with differential tolerance to salt were grown under saline conditions in long barrels for regular root growth to mimic field conditions. Arvanitolia and Lefkolia were previously reported as tolerant to salinity, and Koroneiki and Gaidourelia were characterized as sensitive, exhibiting a decrease in leaf length and leaf area index after 90 days of salinity. Prolyl 4-hydroxylases (P4Hs) hydroxylate cell wall glycoproteins such as arabinogalactan proteins (AGPs). The expression patterns of P4Hs and AGPs under saline conditions showed cultivar-dependent differences in leaves and roots. In the tolerant cultivars, no changes in OeP4H and OeAGP mRNAs were observed, while in the sensitive cultivars, the majority of OeP4Hs and OeAGPs were upregulated in leaves. Immunodetection showed that the AGP signal intensity and the cortical cell size, shape and intercellular spaces under saline conditions were similar to the control in Arvanitolia, while in Koroneiki, a weak AGP signal was associated with irregular cells and intercellular spaces, leading to aerenchyma formation after 45 days of NaCl treatment. Moreover, the acceleration of endodermal development and the formation of exodermal and cortical cells with thickened cell walls were observed, and an overall decrease in the abundance of cell wall homogalacturonans was detected in salt-treated roots. In conclusion, Arvanitolia and Lefkolia exhibited the highest adaptive capacity to salinity, indicating that their use as rootstocks might provide increased tolerance to irrigation with saline water.