Although the diurnal rhythms of affect influence people’s health and behavior, there is a lack of evidence from rural China, where the types and timing of social activities may differ from Western contexts. In this study, a total of 2847 Chinese rural residents from three provinces of China are interviewed using the abbreviated Day Reconstruction Method (DRM) questionnaire. Diurnal rhythms of three affective subjective wellbeing (SWB) indicators—positive affect (PoA), negative affect (NeA), and net affect are analyzed by multilevel models. Our results show PoA and net affect generally increase in magnitude throughout the day with two peaks around noon and in the evening, respectively; whereas, there is an overall decline in NeA as the day passes with two troughs occurring at lunchtime and in the evening. These patterns, however, flatten considerably, with the lunchtime peaks in PoA and net affect (and trough in NeA) disappearing entirely, after further controlling for two social environmental factors—activity type and the quality of social interaction. This study, set in rural China, corroborates the diurnal rhythms of affect from prior Western research to some extent, and highlights that social environmental factors have a significant effect on diurnal rhythms of affect in the rural Chinese context. It is possible that the diurnal rhythms of affect could change in response to stimulation from the environment. Improving some social environmental factors, such as organizing pleasant activities and creating a friendly interactive environment, could contribute to the increase in positive affect and decline in negative affect, thereby enhancing the quality of life.