An inductive oil pollutant detection sensor based on a high-gradient magnetic field structure is designed in this paper, which is mainly used for online detection and fault analysis of pollutants in hydraulic and lubricating oil systems. The innovation of the sensor is based on the inductance detection method. Permalloy is embedded in the sensing region of the sensor, so that the detection area generates a high gradient magnetic field to enhance the detection accuracy of the sensor. Compared with traditional inductive sensors, the sensor has a significant improvement in detection accuracy, and the addition of permalloy greatly improves the stability of the sensor’s detection unit structure. The article theoretically analyzes the working principle of the sensor, optimizes the design parameters and structure of the sensor through simulation, determines the best permalloy parameters, and establishes an experimental system for verification. Experimental results show that when a piece of permalloy is added to the sensing unit, the signal-to-noise ratio (SNR) of iron particles is increased by more than 20%, and the signal-to-noise ratio of copper particles is increased by more than 70%. When two pieces of permalloy are added, the signal-to-noise ratio for iron particles is increased by more than 70%, and the SNR for copper particles is increased several times. This method raises the lower limit of detection for ferromagnetic metal particles to 20 μm, and the lower limit for detection of non-ferromagnetic metal particles to 80 μm, which is the higher detection accuracy of the planar coil sensors. This paper provides a new and faster online method for pollutant detection in oil, which is of great significance for diagnosing and monitoring the health of oil in mechanical systems.