Metal-polymer nanocomposites are hybrid materials combining the superior plasmonic, electrical, and thermal properties of metals with the good elasticity and manufacturability of polymers. This renders metalpolymer nanocomposites promising candidates for conductive filler and coating applications, where mechanical properties are optothermally coupled. Here, we study the interplay of nanostructure, thermoplasmonics, and elastic mechanical properties of silver-polystyrene nanocomposites (AgPS) by transmission electron microscopy, small-angle x-ray scattering, Brillouin light scattering (BLS), and other supplemental techniques. We utilize the well-known particle-brush architecture to ensure a homogeneous and isotropic nanoparticle distribution throughout the hybrid material. The effective longitudinal modulus of the as-prepared samples is found to decrease from 5.7 to 4.8 GPa with increasing Ag content from 0 to 4.4 vol.%. Temperature-dependent BLS measurements reveal the unique contribution of local thermoplasmonic heating that depends on the Ag nanoparticle composition. This thermoplasmonic effect results in a lower apparent glass transition temperature (T g) and a stronger laser power dependence of the speed of sound. Exceeding moderate thermal annealing temperatures (>150 • C) leads to a strong structural rearrangement within the homogeneous nanocomposite material with a peculiar clustering-redispersion effect, which also translates into altered mechanical properties. The annealing-induced Ag nanoparticle aggregation results in an even stronger thermoplasmonic effect. We validate our experimental findings with complementary thermographic measurements and finite-element modeling. Overall, this work demonstrates the combined effects of composition and (reversible) aggregation on the mechanical and thermoplasmonic properties of metal-polymer nanocomposites. It not only deepens our understanding of the interaction between light, temperature, and mechanical properties in metal-polymer nanocomposites but also provides a guide for customizing AgPS nanocomposites for potential applications.