Acute and regular exercise as well as physical activity (PA) is related to well-being and positive affect. Recent studies have shown that even daily, unstructured physical activities increase positive affect. However, the attempt to achieve adherence to PA or exercise in inactive people through public health interventions has often been unsuccessful. Most studies analyzing the activity-affect association in daily life, did not report participants’ habitual activity behavior. Thus, samples included active and inactive people, but they did not necessarily exhibit the same affective reactions to PA in daily life. Therefore the present study investigated whether the association between PA and subsequent affective state in daily life can also be observed in inactive individuals. We conducted a pilot study with 29 inactive university students (mean age 21.3 ± 1.7 years) using the method of ambulatory assessment. Affect was assessed via electronic diary and PA was measured with accelerometers. Participants had to rate affect every 2 h on a six item bipolar scale reflecting the three basic mood dimensions energetic arousal, valence, and calmness. We calculated activity intensity level [mean Metabolic Equivalent (MET) value] and the amount of time spent in light activity over the last 15 min before every diary prompt and conducted within-subject correlations. We did not find significant associations between activity intensity and the three mood dimensions. Due to the high variability in within-subject correlations we conclude that not all inactive people show the same affective reactions to PA in daily life. Analyzing the PA-affect association of inactive people was difficult due to little variance and distribution of the assessed variables. Interactive assessment and randomized controlled trials might help solving these problems. Future studies should examine characteristics of affective responses of inactive people to PA in daily life. General assumptions considering the relation between affect and PA might not be suitable for this target group.