Soft micro-and nanostructures have been extensively developed for biomedical applications. The main focus has been on multifunctional composite materials that combine the advantages of hydrogels and colloidal particles. Magnetic microgels and nanogels can be realized by hybridizing stimuli-sensitive gels and magnetic nanoparticles. They are of particular interest since they can be controlled in a wide range of biological environments by using magnetic fields. In this review, we elucidate physical principles underlying the design of magnetic microgels and nanogels for biomedical applications. Particularly, this article provides a comprehensive and conceptual overview on the correlative structural design and physical functionality of the magnetic gel systems under the concept of colloidal biodevices. To this end, we begin with an overview of physicochemical mechanisms related to stimuli-responsive hydrogels and transport phenomena and summarize the magnetic properties of inorganic nanoparticles. On the basis of the engineering principles, we categorize and summarize recent advances in magnetic hybrid microgels and nanogels, with emphasis on the biomedical applications of these materials. Potential applications of these hybrid microgels and nanogels in anticancer treatment, protein therapeutics, gene therapy, bioseparation, biocatalysis, and regenerative medicine are highlighted. Finally, current challenges and future opportunities in the design of smart colloidal biodevices are discussed.