Holomorphic gauge fields in N = 1 supersymmetric heterotic compactifications can constrain the complex structure moduli of a Calabi-Yau manifold. In this paper, the tools necessary to use holomorphic bundles as a mechanism for moduli stabilization are systematically developed. We review the requisite deformation theory -including the Atiyah class, which determines the deformations of the complex structure for which the gauge bundle becomes non-holomorphic and, hence, non-supersymmetric. In addition, two equivalent approaches to this mechanism of moduli stabilization are presented. The first is an efficient computational algorithm for determining the supersymmetric moduli space, while the second is an F-term potential in the four-dimensional theory associated with vector bundle holomorphy. These three methods are proven to be rigorously equivalent. We present explicit examples in which large numbers of complex structure moduli are stabilized. Finally, higher-order corrections to the moduli space are discussed.