Hybrid organic-inorganic perovskite materials garner enormous attention for a wide range of optoelectronic devices. Due to their attractive optical and electrical properties including high optical absorption coefficient, high carrier mobility, and long carrier diffusion length, perovskites have opened up a great opportunity for high performance photodetectors. This review aims to give a comprehensive summary of the significant results on perovskite-based photodetectors, focusing on the relationship among the perovskite structures, device configurations, and photodetecting performances. An introduction of recent progress in various perovskite structure-based photodetectors is provided. The emphasis is placed on the correlation between the perovskite structure and the device performance. Next, recent developments of bandgap-tunable perovskite and hybrid photodetectors built from perovskite heterostructures are highlighted. Then, effective approaches to enhance the stability of perovskite photodetector are presented, followed by the introduction of flexible and self-powered perovskite photodetectors. Finally, a summary of the previous results is given, and the major challenges that need to be addressed in the future are outlined. A comprehensive summary of the research status on perovskite photodetectors is hoped to push forward the development of this field.