Ferrograph-based wear debris analysis (WDA) provides significant information for wear fault analysis of mechanical equipment. After decades of offline application, this conventional technology is being driven by the online ferrograph sensor for real-time wear state monitoring. However, online ferrography has been greatly limited by the low imaging quality and segmentation accuracy of particle chains when analyzing degraded lubricant oils in practical applications. To address this issue, an integrated optimization method is developed that focuses on two aspects: the structural re-design of the online ferrograph sensor and the intelligent segmentation of particle chains. For enhancing the imaging quality of wear particles, the magnetic pole of the online ferrograph sensor is optimized to enable the imaging system directly observe wear particles without penetrating oils. Furthermore, a light source simulation model is established based on the light intensity distribution theory, and the LED installation parameters are determined for particle illumination uniformity in the online ferrograph sensor. On this basis, a Mask-RCNN-based segmentation model of particle chains is constructed by specifically establishing the region of interest (ROI) generation layer and the ROI align layer for the irregular particle morphology. With these measures, a new online ferrograph sensor is designed to enhance the image acquisition and information extraction of wear particles. For verification, the developed sensor is tested to collect particle images from different degraded oils, and the images are further handled with the Mask-RCNN-based model for particle feature extraction. Experimental results reveal that the optimized online ferrography can capture clear particle images even in highly-degraded lubricant oils, and the illumination uniformity reaches 90% in its imaging field. Most importantly, the statistical accuracy of wear particles has been improved from 67.2% to 94.1%.