Although phase change materials (PCMs) exhibit effective performance in the thermal management of lithium-ion batteries (LIBs), their development is limited by low thermal conductivity and susceptibility to leakage during the solid–liquid phase transition. To address these challenges and enhance thermal management capabilities, this study introduces a novel composite phase change material (CPCM) synthesized by physically mixing paraffin (PA), expanded graphite (EG), and bacterial cellulose (BC). The thermal performance of CPCMs with varying BC proportions is evaluated, and their impact on temperature control in battery thermal management systems (BTMS) is assessed. The results show that the addition of EG and BC significantly improves the thermal conductivity of the CPCM, reaching a value of 1.39 W·m−1·K−1. This also enhances the uniformity of temperature distribution within the battery module and reduces CPCM leakage. By comparing temperature variations within the battery module under different operating conditions, it was found that the intricate network structure of the CPCM promotes uniform temperature distribution, effectively mitigating temperature rise. Consequently, the maximum temperature and maximum temperature difference within the battery module were maintained below 47 °C and 4 °C, respectively. Compared to a system without phase change material at a 3C discharge rate, the maximum cell temperature, maximum module temperature, and maximum temperature difference were reduced by 32.38%, 26.92%, and 34.94%, respectively. These findings provide valuable insights for the design and optimization of BTMS.