Breast cancer incidence in Japanese women has more than tripled over the past two decades. We have previously shown that this marked increase is mostly due to an increase in the estrogen receptor (ER)-positive, HER2-negative subtype. We conducted a case–control study; ER-positive, HER2-negative breast cancer patients who were diagnosed since 2011 and women without disease were recruited. Environmental factors, serum levels of testosterone and 25-hydroxyvitamin D, and common genetic variants reported as predictors of ER-positive breast cancer or found in Asian women were evaluated between patients and controls in pre- and postmenopausal women. To identify important risk predictors, risk prediction models were created by logistic regression models. In premenopausal women, two environmental factors (history of breastfeeding, and history of benign breast disease) and four genetic variants (TOX3-rs3803662, ESR1-rs2046210, 8q24-rs13281615, and SLC4A7-rs4973768) were considered to be risk predictors, whereas three environmental factors (body mass index, history of breastfeeding, and hyperlipidemia), serum levels of testosterone and 25-hydroxyvitamin D, and two genetic variants (TOX3-rs3803662 and ESR1-rs2046210) were identified as risk predictors. Inclusion of common genetic variants and serum hormone measurements as well as environmental factors improved risk assessment models. The decline in the birthrate according to recent changes of lifestyle might be the main cause of the recent notable increase in the incidence of ER-positive breast cancer in Japanese women.