We report on key signal contributions in photothermal deflection spectroscopy (PDS) of semiconductors at photon energies below the bandgap energy and show how to extract the actual absorption properties from the measurement data. To this end, we establish a rigorous computation scheme for the deflection signal including semi-analytic raytracing to analyze the underlying physical effects. The computation takes into account linear and nonlinear absorption processes affecting the refractive index and thus leading to a deflection of the probe beam. We find that beside the linear mirage effect, nonlinear absorption mechanisms make a substantial contribution to the signal for strongly focussed pump beams and sample materials with high two-photon absorption coefficients. For example, the measured quadratic absorption contribution exceeds 5% at a pump beam intensity of about 1.3 × 10 5 W cm 2 ⁄ in Si and at 5 × 10 4 W cm 2 ⁄ in GaAs. In addition, our method also includes thermal expansion effects as well as spatial gradients of the attenuation properties. We demonstrate that these effects result in an additional deflection contribution which substantially depends on the distance of the photodetector from the readout point. This distance dependent contribution enhances the surface related PDS signal up to two orders of magnitude and may be misinterpreted as surface absorption if not corrected in the analysis of the measurement data. We verify these findings by PDS measurements on crystalline silicon at a wavelength of 1550 nm and provide guidelines how to extract the actual attenuation coefficient from the PDS signal.