The well-developed mudstone and gypsum mudstone in the Oligocene Linhe Formation (E3l) in the Hetao Basin are the main source rocks for gypsum. However, the sedimentary environment and organic matter (OM) enrichment factors of E3l are not clear, and this inhibits the prediction of hydrocarbon source rock distribution and resource calculation. Major and trace elements, total organic carbon (TOC), pyrolysis using the Rock-Eval II, and saturated hydrocarbon gas chromatography–mass spectrometry (GC–MS) analyses were performed in this study. The results show that E3l was deposited in brackish water and saline–ultrasaline water, with weak oxidation reduction in an arid and hot environment. Terrestrial input inhibits OM enrichment, while the redox, paleosalinity, paleoclimate, and paleoproductivity play a catalytic role. The main controlling factors of the same lithologic source rocks are different: terrestrial input and paleoclimate have a greater impact on mudstone, and the redox and paleosalinity were more favorable to gypsum mudstone. Although the main controlling factors are different for different lithologies, their OM enrichment characteristics are still consistent. The E3l water body was deep, and the contribution of nutrients from terrigenous debris to OM enrichment was less. In addition, the water retention environment changed significantly during the E3l sedimentary period, resulting in fewer nutrients, which limited the improvement of surface water paleoproductivity. The arid climate increased water evaporation and salinity, which to some extent prevented consumption and decomposition. Weak oxidation-reduction fluctuations and the stratification of the water body were obvious, and this was not only conducive to the enrichment of OM but also to its preservation.