Recently, 2D materials have attracted considerable research attention in nanophotonic and integrated optoelectronic devices due to their intriguing thermal, mechanical, exciton and nonlinear optical response. Despite extensive efforts on various 2D materials (graphene, transition metal dichalcogenides, h-BN, etc.), however, a candidate material with large nonlinear optical (NLO) properties, ultrafast response speed, broadband response window and good thermal stability remains elusive. Black Phosphorus (BP) is a representative 2D semiconductors, which has huge potential for photodetectors, photo-catalysis, thin-film transistors and photodynamic therapy. In this study, we investigated the ultrafast NLO response of black phosphorus quantum dots (BP QDs)/water dispersions via Z-scan measurements with both femtosecond and picosecond laser pulses. We found a sign change of nonlinear refractive index n2 of BP QDs from femtosecond to picosecond timescale. The dynamic response mechanism of BP QDs/water dispersions was studied by using femtosecond transient absorption spectroscopy under 355 nm excitation. A broadband absorption signal was observed ~0.5 ps after pump excitation, and the decay lifetime of this signal was rather slow (several nanoseconds). Our results indicate that BP QDs is a promising NLO material for ultrafast all-optical signal processing applications.