Electron Spin Resonance (ESR) is a powerful spectroscopic technique used to investigate samples with unpaired electrons in a broad range of scientific fields. High-Frequency ESR (HF-ESR) spectrometers operating at sub-THz frequencies are mostly custom-made with non-standard solutions. This paper presents a set of six different exchangeable sample holders with a fast-loading flange for a sub-THz ESR spectrometer operating at high magnetic fields up to 16 T, and temperature ranges of 4 -400 K. Here, we report on the concept, design, and illustrative measurements of non-resonant ESR sample holders for measurements of samples in a liquid solution, polycrystalline compressed powders, oriented single crystals, electrical devices under sub-THz irradiation, as well as for samples transferred from ultrahigh vacuum (UHV) systems without air contamination. Our solution expands the usage possibilities for HF-ESR spectroscopy, showing that one spectrometer with the presented concept of sample holders enables a wide range of applications.