Magnetic flux leakage (MFL) sensors, with their compact configuration and high sensitivity to small defects, have attracted much attention in recent years for the non-destructive testing of ferromagnetic structures. Tunnel magneto-resistive (TMR) devices have superior performances in sensitivity and linear operation range over conventional magneto-resistive devices. In this paper, a commercial TMR device is employed for developing an electromagnet-based MFL sensor. The electromagnet magnetizer includes Helmholtz-like coils together with a custommade magnetic shield. The orthogonal test method is applied to aid the structural parameter optimization to the magnetizer based on the finite element analysis results of magnetic field distribution. In this study a prototype of a TMR-based MFL sensor is developed, and its performances on detecting various types of defects are tested on a scanning apparatus. The experimental results show that the MFL signal induced by a blind hole with dimensions of 0.3 mm in both depth and diameter is detectable. In addition, two adjacent notches located more than 2.0 mm from each other can be clearly distinguished from the received MFL signal. The detectable angular detection range for a single TMR device is estimated as 52°in the tested linear shaft rod. The consistency between the simulated and received MFL signal induced by a row of notches inspires confidence in the proposed sensor design method, which in the future can be transplanted for TMR-based sensor array design. Finally, the TMR-based MFL sensor is used for detecting a flaw of a single broken wire with a diameter of 0.5 mm, and the induced MFL signal can be clearly recognized from the oscillation signal that is generated by the twisted rope surface. Therefore, the presented TMR-based MFL sensor has great potential for steel wire rope inspection with enhanced sensitivity to small defects, and it is capable of being integrated into production lines due to its compact configuration.