A Higgs Factory is considered the highest priority next collider project by the high-energy physics community. Very advanced designs based on radio-frequency cavities exist, and variations on this approach are still being developed. Recently, an option based on electron-bunch driven plasma wakefield acceleration has also been proposed. In this article, we discuss a further option based on proton-driven plasma wakefield acceleration. This option has significant potential advantages due to the high energy of the plasma wakefield driver, simplifying the plasma acceleration stage. Its success will depend on further developments in producing compact high-energy proton bunches at a high rate, which would also make possible a broad range of synergistic particle-physics research.