To ensure the operational reliability of machinery, rolling bearings exposed to complex and poor conditions should be monitored in real-time. Traditional bearing fault diagnosis methods are always dependent on signal analysis and feature extraction, which are complex and time-consuming. Deep learning method exhibits a good ability in extracting the fault feature, while it is limited to noise pollution and insufficient sample data during the training procedure. In this study, a new sparse enhancement neural network based on generalized minimax-concave penalty and convolutional neural network is proposed to capture fault features automatically. To this end, the generalized minimax-concave penalty is first employed to expand the dataset by pollution data denoise and sparse enhancement of the insufficient samples. Second, the amplified dataset is employed to train the fault classification. By employing the datasets of drive end and fan end derived from the Case Western Reserve University (CWRU), a good prediction accuracy can be found in fault diagnosis for rolling bearings.