Motivated by the Lipschitz rigidity problem in scalar curvature geometry, we prove that if a closed smooth spin manifold admits a distance decreasing continuous map of non-zero degree to a sphere, then either the scalar curvature is strictly less than the sphere somewhere or the map is a distance isometry. Moreover, the property also holds for continuous metrics with scalar curvature lower bound in some weak sense. This extends a result in the recent work of Cecchini-Hanke-Schick [6] and answers a question of Gromov. The method is based on studying the harmonic map heat flow coupled with the Ricci flow from rough initial data to reduce the case to smooth metrics and smooth maps so that results by Llarull [23] can be applied.