Voltage-gated ion channels are key players of electrical signaling in cells. As a unique subfamily, voltage-gated proton (Hv) channels are standalone voltage sensors without separate ion conductive pores. Hv channels are gated by both voltage and transmembrane proton gradient (i.e ∆pH), serving as acid extruders in most cells. Amongst their many functions, Hv channels are known for regulating the intracellular pH of human spermatozoa and compensating for the charge and pH imbalances caused by NADPH oxidases in phagocytes. Like the canonical voltage sensors, Hv channels are a bundle of 4 helices (named S1 through S4), with the S4 segment carrying 3 positively charged Arg residues. Extensive structural and electrophysiological studies on voltage-gated ion channels, in general, agree on an outwards movement of the S4 segment upon activating voltage, but the real-time conformational transitions are still unattainable. With purified human voltage-gated proton (hHv1) channels reconstituted in liposomes, we have examined its conformational dynamics, including the S4 segment at different voltage and pHs using single-molecule fluorescence resonance energy transfer (smFRET). Here, we provide the first glimpse of real-time conformational trajectories of the hHv1 voltage sensor and show that both voltage and pH gradient shift the conformational dynamics of the S4 segment to control channel gating. Our results indicate that the S4 segment transits among 3 major conformational states and kinetic analysis suggest that only the transitions between the inward and outward conformations are highly dependent on voltage and pH changes. Our smFRET studies uncover the stochastic conformational dynamics of S4 and demonstrate how voltage and pH shift its conformational distributions to regulate channel gating. Altogether, we propose a kinetic model that explains the mechanisms underlying voltage and pH gating in Hv channels, which may also serve as a general framework for understanding the voltage sensing and gating in other voltage-gated ion channels.