In this chapter, we will highlight conceptual physical approaches towards the fabrication of nanocomposite polymer latexes in which each individual latex particle contains one or more "hard" nanoparticles, such as clays, silicates, titanates, or other metal(oxides). By "physical approaches" we mean that the "hard" nanoparticles are added as pre-existing entities, and are not synthesized in situ as part of the nanocomposite polymer latex fabrication process. We will narrow our discussion to focus on physical methods that rely on the assembly of nanoparticles onto the latex particles after the latex particles have been formed, or its reciprocal analogue, the adhesion of polymer onto an inorganic nanoparticle. First, will discuss the phenomenon of heterocoagulation and its various driving forces, such as electrostatic interactions, the hydrophobic effect, and secondary molecular interactions. We will then address methods that involve assembly of nanoparticles onto or around the more liquid precursors (i.e., swollen/growing latex particles or monomer droplets). We will focus on the phenomenon of Pickering stabilization. We will then discuss features of particle interaction with soft interfaces, and see how the adhesion of particles onto emulsion droplets can be applied in suspension, miniemulsion, and emulsion polymerization. Finally, we will very briefly mention some interesting methods that make use of interface-driven templating for making well-defined assembled clusters and supracolloidal structures.