Waste utilization has gained more and more interest. In this study, crumb rubber and diatomite were used to modify asphalt binder and the short-term aging effects on the properties of modified asphalts were evaluated. Three kinds of modified asphalt were prepared; they are diatomite modified asphalt (DA), crumb rubber modified asphalt (RA), and diatomite and crumb rubber compound modified asphalt (DRA). Thin film oven test (TFOT) was used to simulate short-term aging in the laboratory. Penetration, softening points, ductility, viscosity, elastic recovery, low temperature creep, and rutting susceptibility were tested and analyzed before and after TFOT. The results indicated that the softening point, viscosity, elastic recovery, creep stiffness, and G*/sinδ of DA, RA, and DRA are all linearly increased with increasing age, while the penetration and ductility are linearly decreased with increasing age. High temperature properties of asphalt are obviously improved by the addition of crumb rubber. Low temperature properties of asphalt are affected more seriously by the addition of diatomite. DRA could combine the advantages of diatomite and crumb rubber and achieve better performance in short-term aging resistance than DA and RA.