As one of the key parts of rotary machine, the fault diagnosis and running condition monitoring of rolling bearings are of great importance for normal working and safe production of rotary machine. However, the traditional diagnosis approaches merely count on artificial feature extraction and domain expertise. Meanwhile, the existing convolutional neural networks (CNNs) have the problem of low fault recognition rates. This paper proposes a novel convolutional neural network with one-dimensional structure (ODCNN) for the automatical fault diagnosis of rolling bearings, which adopts six sets of convolutional and max-pooling layers to extract signal features and applies a flattening convolutional layer followed by two fully-connected layers for feature classification. The architectures of one-dimensional LeNet-5, AlexNet, and the proposed ODCNN are illustrated in detail, followed by the obtaining of training and testing samples, which is pre-processed by overlapping the vibration signals of rolling bearings. Finally, the classification experiment is carried out. The experimental results show that the ODCNN has higher fault diagnosis rates and can achieve high accuracy with load variant. Additionally, the extracted features of three CNNs are visualized, which illustrate that the new CNN has a better classification capacity.