Computational trajectory inference enables the reconstruction of cell-state dynamics from single-cell RNA sequencing experiments. However, trajectory inference requires that the direction of a biological process is known, largely limiting its application to differentiating systems in normal development. Here, we present CellRank (https://cellrank.org) for mapping the fate of single cells in diverse scenarios, including perturbations such as regeneration or disease, for which direction is unknown. Our approach combines the robustness of trajectory inference with directional information from RNA velocity, derived from ratios of spliced to unspliced reads. CellRank takes into account both the gradual and stochastic nature of cellular fate decisions, as well as uncertainty in RNA velocity vectors. On data from pancreas development, we show that it automatically detects initial, intermediate and terminal populations, predicts fate potentials and visualizes continuous gene expression trends along individual lineages. CellRank also predicts a novel dedifferentiation trajectory during regeneration after lung injury, which we follow up experimentally by confirming the existence of previously unknown intermediate cell states.