Two hundred years after Malus' discovery of optical anisotropy, the study of polarization‐driven optical effects is as active as ever, generating interest in new phenomena and potential applications. However, in ultrafast optics, the influence of polarization is frequently overlooked being considered as either detrimental or negligible. Here we demonstrate that spatio‐temporal couplings, which are inherent for ultrafast laser systems with chirped‐pulse amplification, accumulate in multi‐pulse irradiation and lead to a strongly anisotropic light‐matter interaction. Our results identify angular dispersion in the focus as the origin for the polarization dependence in modification, yielding an increase in modification strength. With tight focusing (NA ≥ ∼0.4), this non‐paraxial effect leads to a manifestation of spatio‐temporal couplings in photo‐induced modification. We devise a practical way to control the polarization dependence and exploit it as a new degree of freedom in tailoring laser‐induced modification in transparent material. A near‐focus, non‐paraxial field structure analysis of an optical beam provides insight on the origin of the polarization dependent modification. However, single pulse non‐paraxial corrected calculations are not sufficient to explain the phenomena confirming the experimental observations and exemplifying the need for multi‐pulse analysis.