The growing need for the multiband photodetection of a single scene has promoted the development of both multispectral coupling and broadband detection technologies. Photodetectors operating across the infrared (IR) to terahertz (THz) regions have many applications such as in optical communications, sensing imaging, material identification, and biomedical detection. In this review, we present a comprehensive overview of the latest advances in broadband photodetectors operating in the infrared to terahertz range, highlighting their classification, operating principles, and performance characteristics. We discuss the challenges faced in achieving broadband detection and summarize various strategies employed to extend the spectral response of photodetectors. Lastly, we conclude by outlining future research directions in the field of broadband photodetection, including the utilization of novel materials, artificial microstructure, and integration schemes to overcome current limitations. These innovative methodologies have the potential to achieve high-performance, ultra-broadband photodetectors.