This article argues that memories that come from contexts that are adversarial, and that are not always based on communication and sociality, should be better integrated within the existing theories of social memory. Shamans in postsocialist Mongolia claim that previously suppressed origin spirits demand that their descendants become initiated as shamans in exchange for ceasing to harass them for forgetting and abandonment. Some clients refuse to become initiated as shamans and thus choose to sever their relationship with their past. In this article I explore one such refusal, which led to a disintegration of existing social ties, while also yielding unexpected memories. These memories are different from the shared memories that emerge in the context of organized shamanic rituals. Circulated through rumour and supposition instead of positive sociality and sharing, these ‘asocial’ memories also act as a particular kind of ‘poisonous knowledge’, prompting each individual to withdraw from the network as a way of avoiding the alleged harm from unattended spirits. Owing to divergent subject positioning, where one person's remembering is another's forgetting, the haunting by unwanted memories continues, as resolution through unifying communal ritual is not possible.