Images from the Mars Science Laboratory (MSL) mission of lacustrine sedimentary rocks of Vera Rubin ridge on "Mt. Sharp" in Gale crater, Mars, have shown stark color variations from red to purple to gray. These color differences crosscut stratigraphy and are likely due to diagenetic alteration of the sediments after deposition. However, the chemistry and timing of these fluid interactions is unclear. Determining how diagenetic processes may have modified chemical and mineralogical signatures of ancient Martian environments is critical for understanding the past habitability of Mars and achieving the goals of the MSL mission. Here we use visible/near-infrared spectra from Mastcam and ChemCam to determine the mineralogical origins of color variations in the ridge. Color variations are consistent with changes in spectral properties related to the crystallinity, grain size, and texture of hematite. Coarse-grained gray hematite spectrally dominates in the gray patches and is present in the purple areas, while nanophase and fine-grained red crystalline hematite are present and spectrally dominate in the red and purple areas. We hypothesize that these differences were caused by grain-size coarsening of hematite by diagenetic fluids, as observed in terrestrial analogs. In this model, early primary reddening by oxidizing fluids near the surface was followed during or after burial by bleaching to form the gray patches, possibly with limited secondary reddening after exhumation. Diagenetic alteration may have diminished the preservation of biosignatures and changed the composition of the sediments, making it more difficult to interpret how conditions evolved in the paleolake over time. Plain Language Summary Sedimentary rocks found in deserts on Earth often exhibit striking color differences from red and purple to white, which are caused by groundwater dissolving and reprecipitating iron oxides within the rocks. NASA's Mars Science Laboratory (MSL) mission has observed similar color differences on Mars within the sedimentary rocks of Vera Rubin ridge in Gale crater, which were laid down in an ancient lake. We use color images and spectral data from the Mastcam cameras on MSL to investigate the origin of these color differences and find that they are consistent with changes in iron oxides through the ridge. This variation in iron oxides suggests that groundwater flowed through and altered these rocks multiple times before and after they were buried by later sediments. The MSL mission has shown that habitable lake environments once existed in Gale crater, through detections of the building blocks of life, including organic molecules, and by showing that conditions that existed in the lake were clement for life. However, later alteration by groundwater may have diminished the preservation of organics and changed the composition of these rocks, making it more difficult to interpret the details of how conditions evolved in the lake over time.