The average time T required for high-fidelity readout of quantum states can be significantly reduced via a real-time adaptive decision rule. An adaptive decision rule stops the readout as soon as a desired level of confidence has been achieved, as opposed to setting a fixed readout time t f . The performance of the adaptive decision is characterized by the "adaptive-decision speedup," t f =T. In this work, we reformulate this readout problem in terms of the first-passage time of a particle undergoing stochastic motion. This formalism allows us to theoretically establish the maximum achievable adaptive-decision speedups for several physical two-state readout implementations. We show that for two common readout schemes (the Gaussian latching readout and a readout relying on state-dependent decay), the speedup is bounded by 4 and 2, respectively, in the limit of high single-shot readout fidelity. We experimentally study the achievable speedup in a real-world scenario by applying the adaptive decision rule to a readout of the nitrogen-vacancy-center (NV-center) charge state. We find a speedup of ≈2 with our experimental parameters. In addition, we propose a simple readout scheme for which the speedup can, in principle, be increased without bound as the fidelity is increased. Our results should lead to immediate improvements in nanoscale magnetometry based on spin-to-charge conversion of the NV-center spin, and provide a theoretical framework for further optimization of the bandwidth of quantum measurements.