The uncontrolled development of nanoscale roughness during plasma exposure of polymer surfaces is a major issue in the field of semiconductor processing. In this paper, we investigated the question of a possible relationship between the formation of nanoscale roughening and the simultaneous introduction of a nanometer-thick, densified surface layer that is formed on polymers due to plasma damage. Polystyrene films were exposed to an Ar discharge in an inductively coupled plasma reactor with controllable substrate bias and the properties of the modified surface layer were changed by varying the maximum Ar + ion energy. The modified layer thickness, chemical, and mechanical properties were obtained using real-time in situ ellipsometry, x-ray photoelectron spectroscopy, and modeled using molecular dynamics simulation. The surface roughness after plasma exposure was measured using atomic force microscopy, yielding the equilibrium dominant wavelength and amplitude A of surface roughness. The comparison of measured surface roughness wavelength and amplitude data with values of and A predicted from elastic buckling theory utilizing the measured properties of the densified surface layer showed excellent agreement both above and below the glass transition temperature of polystyrene. This agreement strongly supports a buckling mechanism of surface roughness formation.