Agricultural drainage plays an effective role in preventing waterlogging and salinity disasters and also is the main transport pathway for agricultural non-point source pollutants into rivers and lakes. Hence, the water quality of agricultural drainage should be a point of focus. In this paper, nitrogen and phosphorus loss under improved subsurface drainage with different filter materials (gravel, layered sand-gravel, mixed sand-gravel, straw) were studied by a three-year field experiment (2016-2018) compared with the conventional subsurface drainage. The pH values, total nitrogen, ammonia nitrogen, nitrate nitrogen, total phosphorus and soluble reactive phosphate were considered. The results showed that the nitrogen and phosphorus concentrations of drain outflow under improved subsurface drainage with gravel filter were larger than that with layered sand-gravel filter and mixed sand-gravel filter. The improved subsurface drainages with layered sand-gravel filter and mixed sand-gravel filter had an effect on reducing the ammonia nitrogen, total phosphorus and soluble reactive phosphate concentrations of the outflow. Meanwhile, the characteristics of nitrogen and phosphorus loss under the improved subsurface drainage with straw filter were different from that with layered sand-gravel filter and mixed sand-gravel filter. For the improved subsurface drainage with layered sand-gravel filter outflow, the ammonia nitrogen, total phosphorus, and soluble reactive phosphate concentrations were about 13%-78%, 38%-63%, 40%-68% less, and total nitrogen, nitrate nitrogen concentrations were 24%-80%,18%-96% more than that under conventional subsurface drainage. Meanwhile, for the improved subsurface drainage with straw filter outflow, compared with conventional subsurface drainage outflow, the percentage changes of the total nitrogen, nitrate nitrogen, ammonia nitrogen, total phosphorus and the soluble reactive phosphate concentrations were about −76%-62%, −77%-78%, −152%-−274%, −103%-−400% and −221%-−291%, respectively. Additionally, in the outflow of all subsurface drainage patterns, there were much higher total nitrogen and nitrate nitrogen concentrations which should be focused on and the agricultural water management should be adopted.