The purpose of this paper is to compare a classical non-holonomic system-a sphere rolling against the inner surface of a vertical cylinder under gravity-and a class of discrete dynamical systems known as no-slip billiards in similar configurations. A well-known notable feature of the non-holonomic system is that the rolling sphere does not fall; its height function is bounded and oscillates harmonically up and down. The central issue of the present work is whether similar bounded behavior can be observed in the no-slip billiard counterpart. Our main results are as follows: for circular cylinders in dimension 3, the no-slip billiard has the bounded orbits property, and very closely approximates rolling motion, for a class of initial conditions which we call transversal rolling impact. When this condition does not hold, trajectories undergo vertical oscillations superimposed to an overall downward acceleration. Considering cylinders with different cross-section shapes, we show that no-slip billiards between two parallel hyperplanes in Euclidean space of arbitrary dimension are always bounded even under a constant force parallel to the plates; for general cylinders, when the orbit of the transverse system (a concept that depends on a factorization of the motion into transversal and longitudinal components) has period two-a very common occurrence in planar no-slip billiards-the motion in the longitudinal direction, under no forces, is generically not bounded. This is shown using a formula for a longitudinal linear drift that we prove in arbitrary dimensions. While the systems for which we can prove the existence of bounded orbits have relatively simple transverse dynamics, we also briefly explore numerically a noslip billiard system, namely the stadium cylinder billiard, that can exhibit chaotic transversal dynamics.