Response time, an important parameter of the photodetectors, is used to evaluate the ability to follow a fast-changing optical signal. Various strategies, including the utilization of heterojunction and homojunction detectors, have been implemented to improve the response speed. However, high energy barriers restrict the operation of these devices to either zero bias or reverse bias conditions. Herein, a strategy is developed to modulate response speed, in which a two-dimensional (2D) CuBiP 2 Se 6 nanosheet is employed as a novel van der Waals (vdWs) floating gate to enhance the response speed of a 2D PdPSe-based photodetector. The huge difference between the work functions of CuBiP 2 Se 6 and PdPSe enables the built-in field of the contact interface to significantly modulate the surface barrier height of PdPSe, thus changing a Schottky contact to an ohmic contact. Hence, our device yields an enhanced response speed of 148 μs at both forward and reverse bias, faster than the typical PdPSe device by more than 3 orders of magnitude and the CuBiP 2 Se 6 /PdPSe heterostructure device by 3 times. In addition, our device also presents a high responsivity of 370 A/W. Interestingly, imaging and polarized light detection endow our device with broad prospects in future micro-or nano-optoelectronic devices. Our work provides a promising strategy toward nextgeneration ultrafast photodetectors.