The electrodynamic response of ultra-pure materials at low temperatures becomes spatially non-local. This non-locality gives rise to phenomena such as hydrodynamic flow in transport and the anomalous skin effect in optics. In systems characterized by an anisotropic electronic dispersion, the non-local dynamics becomes dependent on the relative orientation of the sample with respect to the applied field, in ways that go beyond the usual, homogeneous response. Such orientational dependence should manifest itself not only in transport experiments, as recently observed, but also in optical spectroscopy. In this paper we develop a kinetic theory for the distribution function and the transverse conductivity of two-and three-dimensional Fermi systems with anisotropic electronic dispersion. By expanding the collision integral into the eigenbasis of a collision operator, we include momentum-relaxing scattering as well as momentum-conserving collisions. We examine the isotropic 2D case as a reference, as well as anisotropic hexagonal and square Fermi-surface shapes. We apply our theory to the quantitative calculation of the skin depth and the surface impedance, in all regimes of skin effect. We find qualitative differences between the frequency dependence of the impedance in isotropic and anisotropic systems. In the latter case, we study the orientational dependence of skin effect, thus providing guidance for the experimental study of non-local optical effects.