We forecast the reionization history constraints, inferred from Lyα damping wing absorption features, for a future sample of ∼20 z 6 gamma-ray burst (GRB) afterglows. We describe each afterglow spectrum by a threeparameter model. First, L characterizes the size of the ionized region (the "bubble size") around a GRB host halo. Second, 〈x H I 〉 is the volume-averaged neutral fraction outside of the ionized bubble around the GRB, which is approximated as spatially uniform. Finally, N H I denotes the column density of a local damped Lyα absorber (DLA) associated with the GRB host galaxy. The size distribution of ionized regions is extracted from a numerical simulation of reionization and evolves strongly across the epoch of reionization (EoR). The model DLA column densities follow the empirical distribution determined from current GRB afterglow spectra. We use a Fisher matrix formalism to forecast the 〈x H I (z)〉 constraints that can be obtained from follow-up spectroscopy of afterglows with S/N = 20 per R = 3000 resolution element at the continuum. We find that the neutral fraction may be determined to better than 10%-15% (1σ) accuracy from this data across multiple independent redshift bins at z ∼ 6-10, spanning much of the EoR, although the precision degrades somewhat near the end of reionization. A more futuristic survey with 80 GRB afterglows at z 6 can improve the precision here by a factor of 2 and extend measurements out to z ∼ 14. We further discuss how these constraints may be combined with estimates of the escape fraction of ionizing photons derived from the DLA column density distribution toward GRBs extracted at slightly lower redshift, z ∼ 5. This combination will help in testing whether we have an accurate census of the sources that reionized the universe.