Face recognition is an active research subject of biometrics due to its significant research and application prospects. The performance of face recognition can be affected by a series of uncontrollable factors, such as illumination, expression, posture and occlusion, which restricts its real-world applications. Therefore, improving the robustness of face recognition to environmental changes became an urgent problem. In this paper, a simplified deep convolutional neural network structure having high robustness under unlimited conditions is designed for face recognition. This structure can improve training speed and face recognition accuracy, and be suitable for small-scale data sets. Inception Module Incorporated Siamese Convolutional Neural Networks (IMISCNN) is developed based on effective reduction of external interference and better features extraction by adopting the Siamese network structure. A cyclical learning rate strategy is also introduced in IMISCNN for better model convergence. Compared to classical face recognition algorithms, such as PCA, PCA and SVM, CNN, PCANet, and the original SNN et al. The accuracy of IMISCNN in CASIA-webface and Extended Yale B standard face database is 99.36% and 99.21%, respectively. Its feasibility and effectiveness have been verified in our experiments. INDEX TERMS Cyclical learning rate, face recognition, inception module, Siamese convolutional neural networks.