The neural generators of the frequency-following response (FFR), a neural response widely used to study the human auditory system, remain unclear. There is evidence that the balance between cortical and subcortical contributions to the FFR varies with stimulus frequency. In this study, we tried to clarify whether this variation extended to subcortical nuclei at higher stimulus frequencies where cortical sources were inactive. We evoked FFRs, in 17 human listeners with normal hearing (9 female), with three complex tones with missing-fundamentals corresponding to musical tones C4 (262 Hz), E4 (330 Hz), and G4 (393 Hz) presented to left, right, or both ears. Source imaging results confirmed the dominance of subcortical activity underlying both fundamental frequency (F0) and second harmonic (H2) components of the FFR. Importantly, several FFR features (spatial complexity, scalp distributions of spectral strength and inter-trial phase coherence, and functional connectivity patterns) varied systematically with stimulus F0, suggesting an unfixed source configuration. We speculated that the variation of FFR source configuration with stimulus frequency resulted from changing relative contributions of subcortical nuclei. Supportively, topographic comparison between the FFR and the auditory brainstem response (ABR) evoked by clicks revealed that the topography of the F0 component resembled that of the click-ABR at an earlier latency when stimulus F0 was higher and that the topography of the H2 component resembled that of the click-ABR at a nearly fixed latency regardless of stimulus F0, particularly for binaurally evoked FFRs. Possible generation sites of the FFR and implications for future studies were discussed.