Abstract-In this paper, rail track irregularity detection system based on computer vision and SVD analysis is proposed and located in the train's operator cabin near the front. Images are captured by FLEA3 camera of PointGrey, and vibration signals are collected by sensor device MPU6050 integrating 3-axis accelerometer and 3-axis gyroscope. Root mean square of gray-scale threshold Pulse Coupled Neural Network (RMS-PCNN) is used for segmentation of the rail track's image in a single loop, and the improved coupled map lattice(CML) is used for filtering the image and signifying the rail track. After perspective, the track radius can be fetched by analysis of regression. Vibration signal filtered by SVD-unscented Kalman filter(UKF) can reflect the wagon movements. In unscented Kalman filter, Cholesky is replaced by SDV in UT(unscented transform), which can solve negative definite matrix caused by covariance matrix on account of calculation error and round-off error. Also numerical stability is improved under the guarantee of filtering accuracy and the same complexity level of algorithm based on SVD-UKF. Looking up the radius record table, the corresponding threshold in gyroscope signal can be selected, and Compared to the super elevation, the invisible irregularity defects of rail bed will be found out.