The condition metric in spaces of polynomial systems has been introduced and studied in a series of papers by Beltrán, Dedieu, Malajovich, and Shub. The interest of this metric comes from the fact that the associated geodesics avoid ill-conditioned problems and are a useful tool to improve classical complexity bounds for Bézout's theorem. The linear case is examined here: using nonsmooth nonconvex analysis techniques, we study the properties of condition geodesics in the space of full rank, real, or complex rectangular matrices. Our main results include an existence theorem for the boundary problem, a differential inclusion for such geodesics based on Clarke's generalized gradients, regularity properties, and a detailed description of a few particular cases: diagonal and unitary matrices. Moreover, we study condition geodesics from a numerical viewpoint, and we develop an effective algorithm that allows us to compute geodesics with given endpoints and helps to illustrate theoretical results and formulate new conjectures.