With accelerated urbanization in China, an increasing number of the migrant elderly following children (MEFC) have appeared. This study aims to explore the effects of living conditions, subjective integration, and social networks on the health-related quality of life (HRQOL) of MEFC in Jinan, China. HRQOL was assessed by the 12-item Short-Form Health Survey, which included the mental component summary (MCS) and the physical component summary (PCS). Univariate analyses and binary logistic regression were used to investigate the association between the above indicators and HRQOL. A total of 656 MEFC were selected by multi-stage cluster random sampling, 25.2% and 25.0% of whom were defined as poor MCS and poor PCS, respectively. Those who understood the local dialect, could trust others, and connected with friends were more likely to have good MCS; those with a nanny, faulty elevator, and no support from their spouse were the reverse. MEFC who were trans-city, had no elevator or a faulty elevator, and went to the hospital alone were more likely to have poor PCS; those who approved of living conditions in their hometowns were the reverse. Results indicated that better living conditions, stronger subjective integration, and wider social networks led to higher HRQOL of MEFC. Implications of the government, communities, and families of MEFC were given to improve their HRQOL.