Traditionally, sulfonated polymers are used as separator materials in PEM fuel cells. Based on recent experimental results on model compounds this paper critically discusses the potentials and limits of sulfonic acid and alternatively phosphonic acid and heterocycles (imidazole) as protogenic groups for PEM fuel cell electrolytes operating at intermediate temperatures (T > 100 °C) and low humidification. Apart from transport properties, the stability and reactivity of mono‐functionalized model compounds (1‐heptylsulfonic acid (S‐C7), 1‐heptylphosphonic acid (P‐C7) and 2‐heptyl‐imidazole (I‐C7)) and a few diphosphonic acids are examined under wet and dry conditions. These are characterized with respect to their proton conductivity (ac impedance spectroscopy), proton diffusion coefficient (pulsed‐field gradient NMR), thermo‐oxidative stability (TGA under air), electrochemical stability (cyclic voltammetry) and their hydration behavior (TGA under water vapor). The sulfonic acid functionalized compound shows reasonable properties only when a minimum hydration level is guaranteed, while phosphonic acid functionalized compounds combine satisfactory proton conductivity even in the water‐free state at intermediate temperatures (T < 200 °C), comparatively high thermo‐oxidative and electrochemical stability and electrochemical reactivity (hydrogen oxidation and oxygen reduction at platinum surfaces). The presence of water leads to moderate water uptake allowing for reasonable conductivities even at room temperature and prevents condensation reactions at higher temperature. The imidazole based system shows the largest electrochemical stability window, but its moderate proton conductivity and thermo‐oxidative stability and the very high overpotential for oxygen reduction on platinum turn out to be severe disadvantages for the envisaged application.