Originally developed for diagnostic ultrasound imaging, piezoelectric transducers are the most widespread technology employed in optoacoustic (photoacoustic) signal detection. However, the detection requirements of optoacoustic sensing and imaging differ from those of conventional ultrasonography and lead to specifications not sufficiently addressed by piezoelectric detectors. Consequently, interest has shifted to utilizing entirely optical methods for measuring optoacoustic waves. All-optical sound detectors yield a higher signal-to-noise ratio per unit area than piezoelectric detectors and feature wide detection bandwidths that may be more appropriate for optoacoustic applications, enabling several biomedical or industrial applications. Additionally, optical sensing of sound is less sensitive to electromagnetic noise, making it appropriate for a greater spectrum of environments. In this review, we categorize different methods of optical ultrasound detection and discuss key technology trends geared towards the development of all-optical optoacoustic systems. We also review application areas that are enabled by all-optical sound detectors, including interventional imaging, non-contact measurements, magnetoacoustics, and non-destructive testing.