Carbon-reinforced polymer composites form an important category of advanced materials, and there is an increasing demand to enhance their performance using more convenient and scalable processes at low costs. In the present study, graphitic flakes were prepared by the mechanical exfoliation of synthetic graphite electrodes and utilized as an abundant and potentially low-cost filler to fabricate epoxy-based composites with different additive ratios of 1–10 wt.%. The morphological, structural, thermal, and mechanical properties of these composites were investigated. It was found that the thermal conductivity of the composites increases by adding graphite, and this increase mainly depends on the ratio of the graphite additive. The addition of graphite was found to have a diverse effect on the mechanical properties of the composites: the tensile strength of the composites decreases with the addition of graphite, whilst their compressive strength and elastic modulus are enhanced. The results demonstrate that incorporating 5 wt% of commercially available graphite into epoxy not only raises the thermal conductivity of the material from 0.223 to 0.485 W/m·K, but also enhances its compressive strength from 66 MPa to 72 MPa. The diverse influence of graphite provides opportunities to prepare epoxy composites with desirable properties for different applications.