We present a numerical study of the effect of an evolving bar on the kinematics of a primordial hot star cluster population in the bulge, as well as on bulge field stars originating within dissolved star clusters. We confirm that an evolving bar is able to transfer a small amount of angular momentum to the star cluster system, in good agreement with observations for Galactic bulge clusters. In the case of a realistic mass model, in which the bar slows down and grows in size, the star cluster escapers evolve in a configuration consistent with that expected for a classical bulge and share comparable kinematic properties with the spheroidal metal-poor component identified within the bulge of the Milky Way. Our results are consistent with a scenario in which the bulge of the Milky Way has a composite nature, with the star clusters belonging and contributing to a primordial pressure-supported component.