Since a single forest walk (Shinrin-yoku or forest bathing) session is reported to improve sleep temporarily, occasional forest walks may have a positive effect on daily sleep. Therefore, this study aimed to examine whether more frequent forest walking is associated with better daily sleep conditions. Data from the second survey of the Japan Multi-Institutional Collaborative Cohort (J-MICC) Daiko Study conducted among residents of Nagoya City, Japan, were used. The study design was a cross-sectional study. In total, 2044 participants (529 men and 1515 women; age, mean ± standard deviation: 58.8 ± 9.9 years) were included in the analysis. Frequent forest walks were associated with a low percentage of insomnia symptoms (Insomnia Severity Index ≥10) in women, but not in men. The adjusted odds ratio for the group that rarely took forest walks with reference to the group that engaged in the activity once a month or more often was 2.04 (95% confidence interval: 1.29–3.23) in women. Forest walk frequency was not significantly associated with sleep duration or sleep efficiency as measured by actigraphy in either men or women. In conclusion, the results suggested that increasing the frequency of forest walks or Shinrin-yoku may be effective in preventing insomnia in women.