White matter (WM) alterations are commonly found across different stages of Alzheimer's disease (AD). However, the association between these changes with underlying AD pathology such as amyloid-β (Aβ) and tau deposition is still poorly understood. Hitherto, most studies have assessed WM alterations in AD using diffusion tensor imaging (DTI). Nonetheless, DTI has methodological shortcomings that limit an accurate biological interpretation. To address this limitation, here we applied fixel-based analysis (FBA) to disentangle microscopic differences in fiber density (FD) from macroscopic morphological changes in fiber cross-section (FC) in early stages of AD. We further investigated the associations of FBA metrics with AD pathology and cognitive performance. Additionally, we compared FBA results with other commonly used WM metrics derived from DTI, free-water corrected (FW)-DTI and diffusion kurtosis imaging (DKI). To achieve these goals, we included 224 Aβ-negative and 91 Aβ-positive cognitively unimpaired individuals as well as 78 Aβ-positive patients with mild cognitive impairment (MCI) with diffusion-weighted MRI (dMRI), Aβ-PET and tau-PET scans from the Swedish BioFINDER-2 study. We found that tau-PET uptake in medial temporal regions was associated with macrostructural alterations reflected by reduced FC mainly in the parahippocampal part of the cingulum bundle in Aβ-positive individuals. This tau-related WM alteration was also associated with impaired memory. Interestingly, only FBA metrics were able to capture the association between tau-PET uptake and white matter degeneration. No association was found between global amyloid load and any dMRI metrics. Compared to both cognitively unimpaired groups, MCI patients showed a decrease in all FBA metrics in the entire cingulum bundle, uncinate fasciculus and anterior thalamic radiations. Metrics derived from DKI, and FW-DTI revealed a similar pattern of alterations whereas the spatial extent of WM abnormalities detected by DTI was more widespread. Altogether, our results indicate that early WM alterations in AD are mainly due to macrostructural changes identified by FBA metrics, being more closely associated with tau than Aβ pathology. These findings suggest that future studies assessing the effects of AD pathology in white matter tracts should consider using FBA metrics.