Red mud, a waste residue of aluminium industry, was used as modified asphalt material to prepare red mud modified asphalt and red mud modified asphalt under freeze-thaw cycles. The matrix asphalt (MA), red mud modified asphalt (RMMA), and red mud modified asphalt under freeze-thaw cycles (RMMAFC) were studied by scanning electron microscopy (SEM), Fourier transform infrared spectrometry (FTIR), atomic force microscopy (AFM), and differential scanning calorimetry (DSC). Microscopic experiments were conducted to investigate the modification performance and mechanism. The modification mechanism of red mud modified asphalt was investigated using molecular dynamics simulation in this study. The results show that red mud can form a uniform and stable blending system with base asphalt after adding base asphalt. The structure of asphalt after adding red mud and adding red mud and freezing-thawing cycles does not change. The bee-structure decreases obviously with the addition of red mud by atomic force microscopy (AFM). Density decreases gradually, but bee-structure height increases obviously; bee-structure of red mud modified asphalt is destroyed after freeze-thaw cycles. Through differential scanning calorimetry (DSC), after adding red mud, heat absorption decreases. Freeze-thaw cycles greatly reduce heat absorption of red mud modified asphalt. Constructing molecular model of major components of red mud (Fe2O3, Al2O3) and asphaltene, simulation results show that the interfacial energy between asphaltene and red mud’s main components Fe2O3 and Al2O3 at −10°C, 25°C, and 170°C is stronger than that of Fe2O3. The results of calculating the interfacial energy of asphaltene on the chemical composition surface of red mud are negative. It can be seen that there are adsorption effects on the surface of asphaltene and red mud. Therefore, increasing the content of Al2O3 or decreasing the content of Fe2O3 in red mud is beneficial to the adsorption of asphaltene.