To improve the durability of pumice lightweight aggregate concrete applied in cold and drought areas, sodium silicate-modified waste tire rubber powder is used to treat the pumice lightweight aggregate concrete. The pumice lightweight aggregate concrete studied is mainly used in river lining structures. It will be eroded by water flow and the impact of ice and other injuries, resulting in reduced durability, and the addition of modified rubber will reduce the damage. The durability, including mass loss rate and relative dynamic elastic modulus of pumice lightweight aggregate concrete with different sodium silicate dosages and rubber power particle sizes, is analyzed under freeze-thaw cycles, and the microstructure is further characterized by using microscopic test methods such as nuclear magnetic resonance tests, ultra-depth 3D microscope tests, and scanning electron microscopy tests. The results showed that the durability of pumice lightweight aggregate concrete is significantly improved by the addition of modified waste tire rubber powder, and the optimum durability is achieved when using 2 wt% sodium silicate modified rubber power with a particle size of 20, and then the mass loss rate decreased from 4.54% to 0.77% and the relative dynamic elastic modulus increased from 50.34% to 64.87% after 300 freeze-thaw cycles compared with other samples. The scanning electron microscopy test result showed that the surface of rubber power is cleaner after the modification of sodium silicate, so the bonding ability between rubber power and cement hydration products is improved, which further improved the durability of concrete under the freeze-thaw cycle. The results of the nuclear magnetic resonance test showed that the pore area increased with the number of freeze-thaw cycles, and the small pores gradually evolved into large pores. The effect of sodium silicate on the modification of rubber power with different particle sizes is different. After the treatment of 2 wt% sodium silicate, the relationship between the increased rate of pore area and the number of freezing-thawing cycles is 23.8/times for the pumice lightweight aggregate concrete containing rubber power with a particle size of 20 and 35.3/times for the pumice lightweight aggregate concrete containing a particle size of 80 rubber power, respectively.