Stability of rider-controlled bicycles is related to the appropriate coupling between leaning and steering. Inspired by this driving experience, we propose a linear control law between steer and lean angles to balance motion of an unmanned bicycle. The selection of the control parameters is based on stability analysis for the controlled bicycle dynamics, in which the control law and the constant rear-wheel velocity ω0 serve as servo-constraints imposed to the bicycle system. To facilitate the stability analysis of the controlled motion, we use tools from geometric mechanics to reduce the bicycle's dynamics, leading to a two-dimensional dynamic system. We thereafter study its relative equilibria and related stability. Theoretical results show that there is a critical value of ωc, above which the bicycle can move in a stable uniform straight forward motion, and below which a pair of relative equilibria related to stable uniform circular motion exist. The driving rule of steering toward a fall (STF) and counter-steering (CST) can also be explained theoretically. Finally, we fabricate a powered autonomous bicycle and propose an error correction algorithm to eliminate the drift error of the gyroscope sensor. Our experimental results are in good agreement with the theoretical predictions. Generally, this paper presents the mathematical basis for designing the more advanced and intelligent autonomous bicycles, and may find potential applications in other vehicle systems.