THz imaging is effective in distinguishing between cancerous, healthy, and fatty tissues in breast tumors, but a challenge remains in the contrast between cancerous and fibroglandular (healthy) tissues. This work investigates carbon-based nanoparticles as potential contrast agents for terahertz imaging of breast cancer. Microdiamonds, nanodiamonds, and nanometer-scale onion-like carbon are characterized with terahertz transmission spectroscopy in low-absorption backgrounds of polydimethylsiloxane or polyethylene. The refractive index and absorption coefficients are calculated based on the measured electric fields. Nanodiamonds show little effect on the terahertz signal, microdiamonds express resonance-like, size-dependent absorption peaks, and onion-like carbon provides a uniform increase in the optical properties even at low concentration. Due to its strong interaction with terahertz frequencies and ability to be activated for selective binding to cancer cells, onion-like carbon is implemented into engineered three-dimensional breast tumor models composed of phantom tissue mimicking infiltrating ductal carcinoma surrounded by a phantom mimicking healthy fibroglandular tissue. This model is imaged using the terahertz reflection mode to examine the effectiveness of contrast agents for differentiation between the two tissue types. In both spectroscopy and imaging, a 10% concentration of onion-like carbon shows the strongest impact on the terahertz signal and holds promise as a terahertz contrast agent.