Background: Currently, sedentariness is assessed over a short period of time, thus it is difficult to study its cognitive implications. To investigate the cognitive consequences of a sedentary lifestyle, the past level (i.e., the sedentary time accumulated over the years) and current level of sedentariness should be considered. This pilot study aimed to investigate the negative association between a sedentary lifestyle and cognition by considering both the current and past sedentariness. It was expected that the physical activity level moderates the potential negative association between sedentariness and cognition. Methods: 52 college students (Mage = 20.19, SDage = 2; 36 women) participated in the study. Current sedentariness (ratio of sedentary time in the last year), past sedentariness (ratio of sedentary time accumulated in previous years), and physical activity (ratio of time spent in physical activity in years) were assessed using a questionnaire. Cognitive inhibition, cognitive flexibility, and working memory updating were measured through three specific tests. Results: Past sedentariness significantly explained the inhibition performance when controlled for physical activity, whereas current sedentariness did not. More precisely, past sedentariness only negatively predicted cognitive inhibition when the physical activity level was low (β = −3.15, z(48) = −2.62, p = 0.01). Conclusions: The impact of sedentariness on cognitive functioning might only be revealed when past sedentariness and physical activity are controlled.