In this paper, the Lipschitz clustering property of a metric space refers to the existence of Lipschitz retractions between its finite subset spaces. Obstructions to this property can be either topological or geometric features of the space. We prove that uniformly disconnected spaces have the Lipschitz clustering property, while for some connected spaces, the lack of sufficiently short connecting curves turns out to be an obstruction. This property is shown to be invariant under quasihomogeneous maps, but not under quasisymmetric ones.