Contact with nature has been used to promote both physical and mental health, and is increasingly used among cancer patients. However, the COVID-19 pandemic created new challenges in both access to nature in public spaces and in cancer care. The purpose of our study was to evaluate the change in active and passive use of nature, places of engaging with nature and associations of nature contact with respect to improvements to perceived stress and symptom experience among breast cancer patients during the pandemic. We conducted a cross-sectional survey of people diagnosed with breast cancer using ResearchMatch (n = 56) in July 2020 (the first wave of COVID-19). In this US-based, predominantly white, affluent, highly educated, female sample, we found that, on average, participants were first diagnosed with breast cancer at 54 years old and at stage 2 or 3. Eighteen percent of participants experienced disruptions in their cancer care due to the pandemic. As expected, activities in public places significantly decreased as well, including use of parks/trails and botanical gardens. In contrast, spending time near home, on the porch or in the backyard significantly increased. Also observed were significant increases in indoor activities involving passive nature contact, such as watching birds through a window, listening to birdsong, and smelling rain or plants. Decreased usage of parks/trails was significantly associated with higher stress (Coef = −2.30, p = 0.030) and increased usage of the backyard/porch was significantly associated with lower stress (Coef = −2.69, p = 0.032), lower symptom distress (Coef = −0.80, p = 0.063) and lower symptom severity (Coef = −0.52, p = 0.009). The most commonly reported alternatives to outdoor engagement with nature were watching nature through a window (84%), followed by looking at images of nature (71%), and listening to nature through a window (66%). The least commonly enjoyed alternative was virtual reality of nature scenes (25%). While outdoor contact with nature away from home decreased, participants still found ways to experience the restorative benefits of nature in and around their home. Of special interest in planning interventions was the fact that actual or real nature was preferred over that experienced through technology. This could be an artifact of our sample, or could represent a desire to be in touch with the “real world” during a health crisis. Nature contact may represent a flexible strategy to decrease stress and improve symptom experience among patients with cancer, particularly during public health crises or disruptions to cancer care.