Two-dimensional (2D) layered materials exhibit strong light-matter interactions, remarkable excitonic effects, and ultrafast optical response, making them promising for high-speed on-chip nanophotonics. Recently, significant attention has been directed towards anisotropic 2D materials (A2DMs) with low in-plane crystal symmetry. These materials present unique optical properties dependent on polarization and direction, offering additional degrees of freedom absent in conventional isotropic 2D materials. In this review, we discuss recent progress in understanding the fundamental aspects and ultrafast nanophotonic applications of A2DMs. We cover structural characteristics and anisotropic linear/nonlinear optical properties of A2DMs, including well-studied black phosphorus and rhenium dichalcogenides, as well as emerging quasi-one-dimensional materials. Then, we discuss fundamental ultrafast anisotropic phenomena occurring in A2DMs, such as polarization-dependent ultrafast dynamics of charge carriers and excitons, their direction-dependent spatiotemporal diffusion, photo-induced symmetry switching, and anisotropic coherent acoustic phonons. Furthermore, we review state-of-the-art ultrafast nanophotonic applications based on A2DMs, including polarization-driven active all-optical modulations and ultrafast pulse generations. This review concludes by offering perspectives on the challenges and future prospects of A2DMs in ultrafast nanophotonics.