The optical Stark effect in a pump-probe setup is expected to show interesting additional features if the quantization of the pump field becomes important. A major effect is that the lineshape of a Stark-shifted resonance is strongly modified when squeezing the pump field. Furthermore, a probe gain is predicted here which does not appear in a semiclassical treatment. It appears for pump detunings considerably lower than the mean Rabi frequency. The nonclassical gain in the optical Stark effect is investigated on an ensemble of two-level systems (TLSs). A density matrix approach is presented which allows the accurate calculation of the probe absorption. The precise treatment of correlations between pairs of TLSs is crucial to explain details of the lineshape.