As typical representatives of group III chalcogenides, InSe, α-In2Se3, and β′-In2Se3 have drawn considerable interest in the domain of photoelectrochemistry. However, the microscopic mechanisms of carrier dynamics in these systems remain largely unexplored. In this work, we first reveal that hot electrons in the three systems have different cooling rate stages and long-lived hot electrons, through the utilization of density functional theory calculations and nonadiabatic molecular dynamics simulations. Furthermore, the ferroelectric polarization of α-In2Se3 weakens the nonadiabatic coupling of the nonradioactive recombination, successfully competing with the narrow bandgap and slow dephasing process, and achieving both high optical absorption efficiency and long carrier lifetime. In addition, we demonstrate that the ferroelectric polarization of α-In2Se3 not only enables the formation of the double type-II band alignment in the InSe/α-In2Se3/InSe heterostructure, with the top and bottom InSe sublayers acting as acceptors and donors, respectively, but also eliminates the hindrance of the built-in electric field at the interface, facilitating an ultrafast interlayer carrier transfer in the heterojunction. This work establishes an atomic mechanism of carrier dynamics in InSe, α-In2Se3, and β′-In2Se3 and the regulatory role of the ferroelectric polarization on the charge carrier dynamics, providing a guideline for the design of photoelectronic materials.