In spite of the promising properties as polymer electrolytes for fuel cells, H3PO4‐doped PBI systems need to be further optimised for what concerns the oxidative stability, the mechanical performances and the leaching of acid at temperatures lower than 150 °C. To this aim, the preparation of composite and nanocomposite membranes is of interest. Here, we investigate on the actual role of the filler in determining the physical and chemical properties of the membranes. Three types of SiO2 are compared, namely HiSil™ T700, mesoporous SBA‐15 and an imidazole‐functionalised silica, which differ for morphology, microstructure and chemical nature. Particular attention is devoted to the ability of the filler to reduce the drop of conductivity occurring in consequence of the acid leaching. In the as‐doped systems, the proton transport is remarkably improved by the presence of fillers containing active sites for the proton hopping. In the case of washed membranes the acid retention capability and the permanent proton conductivity are increased by fillers with higher surface area or with higher basicity. In the lights of these results, the imidazole‐derivatised silica seems to be an optimal filler for the preparation of PBI membranes for PEMFCs operating at 120 °C.