Auto-driving detection usually acquires low-light infrared images, which pose a great challenge to the autopilot function at night due to their low contrast and unclear texture details. As a precursor algorithm in the field of automatic driving, the infrared image contrast enhancement method is of great significance in accelerating the operation speed of automatic driving target recognition algorithms and improving the accuracy of object localization. In this study, a convolutional neural network model including feature extraction and image enhancement modules is proposed to enhance infrared images. Specifically, the feature extraction module consists of three branches, a concatenation layer, and a fusion layer that connect in parallel to extract the feature images. The image enhancement module contains eight convolutional layers, one connectivity layer, and one difference layer for enhancing contrast in infrared images. In order to overcome the problem of the lack of a large amount of training data and to improve the accuracy of the model, the brightness and sharpness of the infrared images are randomly transformed to expand the number of pictures in the training set and form more sample pairs. Unlike traditional enhancement methods, the proposed model directly learns the end-to-end mapping between low- and high-contrast images. Extensive experiments from qualitative and quantitative perspectives demonstrate that our method can achieve better clarity in a shorter time.