Serious mental illness (SMI) can significantly impact the lives of individuals and their families. These families often experience great emotional distress over time due to the early onset of SMI, which in turn leads to long-term trajectories and only partial recovery. However, we do not fully understand the emotional distress of family caregivers. Thus, our aim was to enrich the understanding of the lived experiences of family caregivers’ emotional trajectories of distress while caring for persons with SMI. We conducted a secondary analysis using a hermeneutic approach to the narratives of seven family caregivers from a study on living with voices unheard by others. Participants’ trajectories of emotional distress came forth as being thrust on an unpredictable, intensely worrisome, and indefinite journey. The following themes highlighted this tumultuous journey: fumbling in the dark trying to grasp the incomprehensible, “on your toes”—enduring unpredictability, facing different forms of fear, and battling waves of sadness and regret. Caregivers face multiple threats to their well-being and sometimes even to their health. Their distress appeared to vary according to their relationship with the person with SMI, whether they lived with the ill person, illness trajectory, and amount of violent or suicidal behavior. The results underscore the need for individualized and timely information, opportunities for dialogue with healthcare providers with and without the person with SMI, and inclusion in care planning. Caregivers who have experienced trauma, threats of violence, and rejection require special attention.