Lower Paleozoic sedimentary rocks in the northwestern Tarim basin were strongly altered by complicated geofluids, which resulted in the occurrence of various diagenetic minerals (e.g., dolomite). Here, in situ major, trace, and rare earth element geochemistry of Lower Ordovician diagenetic dolomite grains as well as petrography were performed to unravel the geochemical features, the nature, and origin of the diagenetic fluids. The results indicate that different geochemical information can be detected within a single sample, even within a single dolomite grain. Five generations of diagenetic dolomite have been identified based on geochemical signatures, resulting from four distinct types of diagenetic fluids: (1) HREE enrichment (PAAS-normalized), low RREE, no Eu anomaly, low Mn, Ba, moderate Fe, and high Sr contents are probably due to early burial dolomitizing fluids; (2) MREE enrichment, high RREE, high Mn, Fe, and low Sr content are likely to be associated with Devonian deep-circulating crustal hydrothermal fluids; (3) flat or LREE enrichment pattern with obviously positive Eu anomaly is inferred to be linked to Permian magmatic hydrothermal fluids; and (4) flat REE pattern, moderate RREE, no Eu anomaly, low Mn, Ba, moderate Fe, and high Sr contents are probably due to late burial dolomitizing fluids. The significances of in situ method demonstrated in this study, compared with the whole rock analysis, include not only contamination-free analysis but also unraveling the internal geochemical variation within a single sample or a mineral grain. Thus, for the geochemical study of complicated diagenetic process, in situ method should be preferentially considered.