The hot carriers generated from the nonradiative decay of surface plasmons in metallic nanostructures can inject into the conduction band of a semiconductor, allowing for the sub‐bandgap photodetection under room temperature. By the controllable interfacial barrier height between the plasmonic and semiconductor/insulator materials, the hot carrier photodetectors working from ultraviolet to infrared bands are extensively demonstrated with significant progress. In this review, hot carrier dynamics are briefly discussed from generation, transport, and emission perspectives. The state‐of‐the‐art progress of hot carrier photodetectors with various configurations, material constitutions, and plasmonic nanostructures are surveyed. To further promote hot carrier extraction efficiency toward the practical applications, the thermodynamic loss analysis, and the potential strategies from the optical, electrical, and material perspectives are addressed. The performances of the developed hot carrier photodetectors are also summarized, particularly addressing the novel functionalities, challenges, and future opportunities.