Humans learn and process language in complex and dynamic contexts, involving simultaneous processing of connected speech, faces, bodies, objects and including, at times, touch, odor, and taste. How words and their associated concepts are encoded in the brain during real world processing is unclear. We investigated the representational structure of conceptual encoding in a naturalistic setting and asked to what extent brain responses dynamically depend on context. Across different contexts, we find evidence that concrete and abstract concepts encode experience-based information in roughly differential sets of brain regions. However, these differences reduce when contextual variance is considered. Specifically, the response profile of abstract concepts becomes more concrete-like when they are processed in context highly related to their meaning. Conversely, when context is unrelated to a given concrete concept, the activation pattern resembles more that of abstract conceptual processing. Our results suggest that while concepts encode habitual experiences on average, the underlying neurobiological organization is not fixed but depends dynamically on available contextual information at the time of processing.