Abiotic stresses could have complex and diverse effects on the growth and development of forest trees. In this review, we summarized the responses of morphological, physiological and anatomical traits in woody plants to abiotic stresses, including drought, flood, extreme temperature, salinity, heavy metal, microplastics and combined stresses, especially from the xylem perspective. Under most abiotic stress, xylem hydraulic conductivity decreases, which is associated with leaf stomatal regulation and the inhibition of aquaporin (AQP) activity. Meanwhile, woody plants regulate the size and morphology of their roots and leaves to balance water absorption and transpiration. The anatomical traits are also altered, such as denser leaf stomata, narrower conduits and thicker cell walls. In addition, different stresses have unique effects, such as flood-induced adventitious roots and aeration tissues, forest fire-induced irreversible xylem damage, low temperature-induced tissue freezing, salt stress-induced hinderance of ion absorption and heavy metal-induced biological toxicity. Under stresses of drought, flooding and heavy metals, woody plants’ growth may occasionally be promoted. The effects of combined stress on the physiological, morphological and anatomical traits of woody plants are not simply additive, with the related mechanism to be further studied, especially in natural or near-natural conditions.