We study random walks on groups with the feature that, roughly speaking, successive positions of the walk tend to be "aligned". We formalize and quantify this property by means of the notion of deviation inequalities. We show that deviation inequalities have several consequences including Central Limit Theorems, the local Lipschitz continuity of the rate of escape and entropy, as well as linear upper and lower bounds on the variance of the distance of the position of the walk from its initial point. In a second part of the paper, we show that the (exponential) deviation inequality holds for measures with exponential tail on acylindrically hyperbolic groups. These include nonelementary (relatively) hyperbolic groups, Mapping Class Groups, many groups acting on CAT(0) spaces and small cancellation groups.