Infrared photodetectors are gaining remarkable interest due to their widespread civil and military applications. Low-dimensional materials such as quantum dots, nanowires, and two-dimensional nanolayers are extensively employed for detecting ultraviolet to infrared lights. Moreover, in conjunction with plasmonic nanostructures and plasmonic waveguides, they exhibit appealing performance for practical applications, including sub-wavelength photon confinement, high response time, and functionalities. In this review, we have discussed recent advances and challenges in the prospective infrared photodetectors fabricated by low-dimensional nanostructured materials. In general, this review systematically summarizes the state-of-the-art device architectures, major developments, and future trends in infrared photodetection.