Restrictive COVID-19 measures can have significant mental health impacts, particularly on young people. How such measures may influence day-to-day momentary affect, nonetheless, remains to be explored. Experience sampling data were collected from 165 young people (aged 15–24) as part of a larger epidemiological youth mental health study in Hong Kong. We examined the impact of one of the most stringent COVID-19 measures – dine-in restrictions – on momentary positive and negative affect and current contexts and activities of these young people. The effects of a milder form of COVID-19 measure – school suspension – were separately examined. Multilevel analysis revealed that those in the dine-in ban group, compared to dining-as-usual, showed significantly reduced momentary positive affect ( β = −0.17, SE = 0.06, p = 0.003). Its effect remained significant even when accounting for baseline depressive and anxiety symptoms and socioeconomic status ( β = −0.15, SE = 0.05, p = 0.008). The effect of dine-in ban on reduced momentary positive affect was found specifically when participants were in indoor locations (e.g., home, office), alone, and engaged in passive leisure activities. This pattern was not observed when participants were at school or at other outdoor locations, with friends, or engaged in active leisure activities. No significant effect of school suspension on momentary affect was observed. More severe COVID-19 measures, such as dine-in ban, can have significant impacts on the momentary positive affect of young people. Certain contexts and activities may offer protection against the consequences of COVID-19 measures. The current findings may help to inform future designs of mental health interventions and public health policies. Supplementary Information The online version contains supplementary material available at 10.1007/s12144-022-03183-y.