Our recent paper reports the experimental realization of a one-atom laser in a regime of strong coupling [J. McKeever, A. Boca, A. D. Boozer, J. R. Buck, and H. J. Kimble, Nature (London) 425, 268 (2003)]. Here we provide the supporting theoretical analysis relevant to the operating regime of our experiment. By way of a simplified four-state model, we investigate the passage from the domain of conventional laser theory into the regime of strong coupling for a single intracavity atom pumped by coherent external fields. The four-state model is also employed to exhibit the vacuum-Rabi splitting and to calculate the optical spectrum. We next extend this model to incorporate the relevant Zeeman hyperfine states as well as a simple description of the pumping processes in the presence of polarization gradients and atomic motion. This extended model is employed to make quantitative comparisons with our earlier measurements for the intracavity photon number versus pump strength and for the photon statistics as expressed by the intensity correlation function g ͑2͒ ͑͒.