The use of smart colloidal microgels for advanced applications critically depends on their response kinetics. We use pressure jump small angle neutron scattering with supreme time resolution to study the rapid volume phase transition kinetics of such microgels. Utilizing the pressure induced microphase separation inside the microgels we were able to resolve their collapse and swelling kinetics. While the collapse occurs on a time scale of 10 ms, the particle swelling turned out to be much faster. Photon correlation spectroscopy and static small angle neutron scattering unambiguously show, that the much slower collapse can be associated with the complex particle architecture exhibiting a loosely-crosslinked outer region and a denser inner core region. These insights into the kinetics of stimuli-responsive materials are of high relevance for their applications as nano-actuators, sensors or drug carriers. Moreover, the used refined pressure jump small angle neutron scattering technique is of broad interest for soft matter studies.