Responsive hybrid nanogels, comprising inorganic nanoparticles (NPs) immobilized in a polymer nanogel, have recently attracted intensive interest. While inorganic NPs exhibit size-, shape-, and interdistance-dependent optical, magnetic, electronic, and catalytic properties, responsive polymer nanogels can undergo a volume phase transition in response to external stimuli, such as temperature, pH, glucose level, light, and other stimulus, to modify the physicochemical environment of the inorganic NPs immobilized inside. The controllable properties of the responsive hybrid nanogels can offer the possibilities for external switching and manipulation when applied to a diverse range of applications. In this Chapter we provide an analysis of recent advances on the responsive polymer-inorganic hybrid nanogels. The Chapter consists of three major parts: the fundamental theory, synthesis, and applications of the polymer-inorganic hybrid nanogels in optical sensing, diagnostic imaging, and drug delivery.