In this paper, an eddy current testing system equipped with low-performance processor is designed for rail defect detection. A digital lock-in amplifier (DLIA) based on adaptive Kalman filter (AKF) is presented to detect the weak voltage induced in two differential coils. The proposed method can fast demodulate the amplitude of a signal with a randomized phase using one cycle of the sinusoidal signal, and simultaneously improve signal-to-noise ratio. This DLIA has made use of the sinusoidal orthogonality and concurrently utilized AKF to track on the amplitude variation of voltage, which is numerically simulated and analyzed in MATLAB. The simulated results indicate that increasing excitation frequency or sampling points in one cycle has an effective suppression in low-frequency noise. Furthermore, the experiment is carried out to verify the performance of the system for rail defect detection. Finally, the experimental result shows that the proposed lock-in amplifier has a root mean squared error of 1.3 mV and performs well in terms of memory consumption and precision. Moreover, the amplitude variation in differential coils is linearly tracked, and the surface defects on rail specimens can be detected.