Context. Gaia is a space mission that is currently measuring the five astrometric parameters, as well as spectrophotometry of at least 1 billion stars to G = 20.7 mag with unprecedented precision. The sixth parameter in phase space (i.e., radial velocity) is also measured thanks to mediumresolution spectroscopy that is being obtained for the 150 million brightest stars. During the commissioning phase, two fields, one around each ecliptic pole, have been repeatedly observed to assess and to improve the overall satellite performances, as well as the associated reduction and analysis software. A ground-based photometric and spectroscopic survey was therefore initiated in 2007, and is still running to gather as much information as possible about the stars in these fields. This work is of particular interest to the validation of the radial velocity spectrometer outputs. Aims. The paper presents the radial velocity measurements performed for the Southern targets in the 12−17 R magnitude range on high-to mid-resolution spectra obtained with the GIRAFFE and UVES spectrographs. Methods. Comparison of the South Ecliptic Pole (SEP) GIRAFFE data to spectroscopic templates observed with the HERMES (Mercator in La Palma, Spain) spectrograph enabled a first coarse characterisation of the 747 SEP targets. Radial velocities were then obtained by comparing the results of three different methods. Results. In this paper, we present an initial overview of the targets to be found in the 1 sq. deg SEP region that was observed repeatedly by Gaia ever since its commissioning. In our representative sample, we identified one galaxy, six LMC S-stars, nine candidate chromospherically active stars, and confirmed the status of 18 LMC Carbon stars. A careful study of the 3471 epoch radial velocity measurements led us to identify 145 RV constant stars with radial velocities varying by less than 1 km s −1 . Seventy-eight stars show significant RV scatter, while nine stars show a composite spectrum. As expected, the distribution of the RVs exhibits two main peaks that correspond to Galactic and LMC stars. By combining [Fe/H] and log g estimates, and RV determinations, we identified 203 members of the LMC, while 51 more stars are candidate members. Conclusions. This is the first systematic spectroscopic characterisation of faint stars located in the SEP field. During the coming years, we plan to continue our survey and gather additional high-and mid-resolution data to better constrain our knowledge on key reference targets for Gaia.