In US cities, neighborhoods have long been racially segregated. However, people do not spend all their time in their neighborhoods, and the consequences of residential segregation may be tempered by the contact people have with other racial groups as they traverse the city daily. We examine the extent to which people’s regular travel throughout the city is to places “beyond their comfort zone” (BCZ), i.e., to neighborhoods of racial composition different from their own—and why. Based on travel patterns observed in more than 7.2 million devices in the 100 largest US cities, we find that the average trip is to a neighborhood less than half as racially different from the home neighborhood as it could have been given the city. Travel to grocery stores is least likely to be BCZ; travel to gyms and parks, most likely; however, differences are greatest across cities. For the first ~10 km people travel from home, neighborhoods become increasingly more BCZ for every km traveled; beyond that point, whether neighborhoods do so depends strongly on the city. Patterns are substantively similar before and after COVID-19. Our findings suggest that policies encouraging more 15-min travel—that is, to amenities closer to the home—may inadvertently discourage BCZ movement. In addition, promoting use of certain “third places” such as restaurants, bars, and gyms, may help temper the effects of residential segregation, though how much it might do so depends on city-specific conditions.