We revisit existing criticism of the Free Energy Principle and Active Inference through the concept of cognitive contextuality. Contextuality describe the property of physical states which are brought about by the very act of their observation, such as the position or momentum of individual particles in quantum settings. Based on conceptual and physical argument, we propose that contextuality drives the construction of cognitive semiotics. As such, it constitutes a fundamental component of cognition which any formal theory thereof must capture. At a conceptual level, Active Inference seems to capture the inherently contextual nature of cognitive engagement with the world. However, the Free Energy Principle cannot formalize this intuition due to its definition in terms of Dynamical Systems Theory, which comes with a well-defined space of possible states with no contextual properties. We describe the duality between agent-driven interaction and the construction of cognitive spaces which is implicit in the very concept of cognitive contextuality. In consequence, we emphasize the importance of the ontic regime called "Participatory Realism", and consider some ways that the FEP could integrate it.