The refractive index modulation (Δn) of photopolymers is a key performance indicator in determining their application potentials in holographic optical elements (HOEs). In this study, four fluoroelastomers (HT-HS, HT-F185, HT-L80, and HT-F197) containing hydroxyl groups as active functional groups are selected to construct low refractive index host matrices with isocyanates. Through a combination of them with high refractive index monomers, a series of transparent photopolymers are obtained. Among them, one containing 16 wt % HT-F197 has the highest sensitivity, which can record a transmission volume holographic grating (VHG) with high resolution (3600 lines/mm) and diffraction efficiency (95.76%) under exposure of a 532 nm laser. Its Δn is up to 0.054. Furthermore, a reflection VHG with high performance (4900 lines/mm resolution and 92.32% diffraction efficiency) can also be achieved. After photobleaching, its transmittance within 400−800 nm reaches 96.21%, showing great potential in the fabrication of transparent HOEs.