The breakup of inertial, solid aggregates in an incompressible, homogeneous and isotropic three-dimensional turbulent flow is studied by means of a direct numerical simulation, and by a Lagrangian tracking of the aggregates at varying Stokes number and fluid-to-particle density ratio. Within the point-particle approximation of the Maxey-Riley-Gatignol equations of motion, we analyse the statistics of the time series of shear and drag stresses, which are here both deemed as responsible for particle breakup. We observe that, regardless of the Stokes number, the shear stresses produced by the turbulent velocity gradients similarly impact the breakup statistics of inertial and neutrally buoyant aggregates, and dictate the breakup rate of loose aggregates. When the density ratio is different from unity, drag stresses become dominant and are seen to be able to cause to breakup of also the most resistant aggregates. The present work paves the way for including the role of inertia in population balance models addressing particle breakup in turbulent flows.