We present a unified approach for constructing Slepian functions -also known as prolate spheroidal wave functions -on the sphere for arbitrary tensor ranks including scalar, vectorial, and rank 2 tensorial Slepian functions, using spin-weighted spherical harmonics. For the special case of spherical cap regions, we derived commuting operators, allowing for a numerically stable and computationally efficient construction of the spin-weighted spherical-harmonic-based Slepian functions. Linear relationships between the spin-weighted and the classical scalar, vectorial, tensorial, and higher-rank spherical harmonics allow the construction of classical spherical-harmonic-based Slepian functions from their spin-weighted counterparts, effectively rendering the construction of spherical-cap Slepian functions for any tensorial rank a computationally fast and numerically stable task.