Background: PET imaging studies have shown that spatially distributed measurements of β-amyloid are significantly correlated with glucose metabolism in Mild Cognitive Impairment (MCI) independently of the APOE ε4 genotype. In contrast, the relationship between tau and glucose metabolism at different stages of Alzheimer's Disease (AD) has not been fully understood. Objective: We hypothesize that spatially distributed scores of tau PET are associated with an even stronger reduction of glucose metabolism, independent of the APOE ε4 genotype and gradually modulated by β-amyloid. Methods: We applied a cross-sectional statistical analysis to concurrent [18F]flortaucipir PET, [18F]florbetapir PET, and 2-[18F]fluoro-2-deoxyglucose (FDG) PET images from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. We employed a Singular Value Decomposition (SVD) approach to the cross-correlation matrix between tau and the FDG images, as well as between tau and β-amyloid PET images. The resulting SVD-based tau scores are associated with cortical regions where a reduced glucose metabolism is maximally correlated with distributed patterns of tau, accounting for the effect of spatially distributed β-amyloid. Results: From a population of MCI subjects, we found that the SVD-based tau scores had their maximal spatial representation within the entorhinal cortex and the lateral inferior temporal gyrus, and were significantly correlated with glucose metabolism in several cortical regions, independently from the confounding effect of the β-amyloid scores and APOE ε4. Moreover, β-amyloid gradually modulated the association between tau and glucose metabolism. Conclusions: Our approach uncovered spatially distributed patterns of the tau-glucose metabolism relationship after accounting for the β-amyloid effects. We showed that the SVD-based tau scores have a strong relationship with decreasing glucose metabolism. By highlighting the more significant role of tau, rather than β-amyloid, on the reduction of glucose metabolism, our results could have important consequences in the therapeutic treatment of AD.