Past research tells us that individuals can infer information about a target's emotional state and intentions from their facial expressions (Frith & Frith, 2012), a process known as mentalising.This extends to inferring the events that caused the facial reaction (e.g. Pillai, Sheppard, & Mitchell, 2012;Pillai et al., 2014), an ability known as retrodictive mindreading. Here, we enter new territory by investigating whether or not people (perceivers) can guess a target's social context by observing their response to stimuli. In Experiment 1, perceivers viewed targets' responses and were able to determine whether these targets were alone or observed by another person. In Experiment 2, another group of perceivers, without any knowledge of the social context or what the targets were watching, judged whether targets were hiding or exaggerating their facial expressions; and their judgments discriminated between conditions in which targets were observed and alone. Experiment 3 established that another group of perceivers' judgments of social context were associated with estimations of target expressivity to some degree. In Experiments 1 and 2, the eye movements of perceivers also varied between conditions in which targets were observed and alone. Perceivers were thus able to infer a target's social context from their visible response. The results demonstrate an ability to use other minds as a window onto a social context that could not be seen directly.