Cold Bitumen Emulsion (CBE) mixture technologies have been recently developed to lower pavement construction temperatures to reduce environmental costs and control gas emissions. Due to its poor early mechanical strength, active fillers (i.e., cement) have been used to obtain high early stiffness in order to have the potential for timely construction of the next layer. There is, however, a lack of understanding about the impact of active fillers on the viscoelastic behavior and fatigue damage resistance of CBE mastics. This study, therefore, aims to identify the influence of active fillers on the rheological properties and the resulting fatigue behavior of CBE mastic, supported by chemical analysis for the filler-bitumen emulsion. For this aim, bitumen emulsion was mixed separately with seven fillers/blended fillers to prepare the CBE mastics. Various experiments, including continuous pH monitoring tests (chemical reactivity of filler-bitumen emulsion), Strain Sweep (SS) tests, Temperature-Frequency Sweep (TFS) tests, Time Sweep (TS) tests, and Linear Amplitude Sweep (LAS) tests were conducted on the CBE binder and the prepared mastics. Results show that the rheological performance and the fatigue damage resistance depend not only on the filler inclusions but also on filler type and chemistry. On this basis, the rise in complex shear modulus and the decrease in the viscous component is associated with a significant enhancement in fatigue performance for specific fillers.