The use of the Internet has a promoting effect on the physical health of the older people. However, previous studies are mostly focused on the perspective of the overall population, or limited to the direct effects, ignoring the exploration of the mechanism of action and the perspective of the older people. Based on the data of the China Family Panel Survey (CFPS) in 2014, 2016, 2018, and 2020, this study found that the use of the Internet has a significant effect on the physical health of the older people, especially among the population groups of females, rural residents, and those living in central and western regions of China. In addition, this study also found that the use of the Internet by the older people can increase their exercise frequency, thereby improving their physical health. Therefore, to promote active aging, this study proposes to further increase the popularity of the Internet among the older people, encourage the introduction of age-appropriate Internet systems and sports facilities, create an online fitness platform for the older people, and promote scientific fitness programs for them.