The field of comparative cognition is intent on demarcating cognition into social and physical domains; however, we argue that it is not useful to differentiate between cognitive processes involving social versus physical (nonsocial) phenomena. We argue that similar cognitive mechanisms underly reasoning about social and physical information and that it is transfer of knowledge and skills within and across these "domains" that is most informative with regard to identifying cognitive flexibility. We argue that social complexity, typically defined as group size, although important, has no special influence over the evolution of social cognition. Likewise, foraging complexity, typically conceptualized as dietary breadth, extractive foraging, and patchy food distribution, has no special influence over the evolution of physical cognition. Instead, researchers should treat social and foraging complexity as continuous rather than dichotomous variables that interact with one another and with environmental unpredictability to predict domain-general cognition that can be applied flexibly across novel features, contexts, and experiences. To answer long-standing questions about the selection pressures for cognition, researchers must consider species that differ in various aspects of their ecology on analogous tasks within and across domains.