In this study, the torsional fretting fatigue experiments are carried out on a multiaxis fatigue testing machine under a sinusoid torque to investigate the fretting damage mechanism of 316L austenitic stainless steel. The S-N curve of torsional fretting fatigue displays different characteristics than that of the plain fatigue curve. After test, the fretting damage zones are analyzed in detail using scanning electron microscopy, electron probe microanalysis, and transmission electron microscopy. The transmission electron microscopic analyses show that there are two types of evolution in the dislocation structures: deformation twin system and cellular structure. With the increase in the torsional stress amplitudes, the dislocation structure changes from deformation twin system into walls-and-channel structures, finally evolving into cellular structures.