Extremal dynamics is the mechanism that drives the BakSneppen model into a (self-organized) critical state, marked by a singular stationary probability density p(x). With the aim of understanding of this phenomenon, we study the BS model and several variants via mean-field theory and simulation. In all cases, we find that p(x) is singular at one or more points, as a consequence of extremal dynamics. Furthermore we show that the extremal barrier x i always belongs to the 'prohibited' interval, in which p(x) = 0. Our simulations indicate that the Bak-Sneppen universality class is robust with regard to changes in the updating rule: we find the same value for the exponent π for all variants. Mean-field theory, which furnishes an exact description for the model on a complete graph, reproduces the character of the probability distribution found in simulations. For the modified processes mean-field theory takes the form of a functional equation for p(x).