End-to-end single-lens imaging system design is a method to optimize both optical system and reconstruction algorithm. Most end-to-end single lens systems use convolutional neural networks (CNN) for image restoration, which fit the transformation relationship between the aberrated image and the ground truth image in the training set. Based on the principle of optical imaging, we realize non-blind image restoration through Wiener deconvolution. Wiener deconvolution is improved with the powerful fitting ability of depth learning so that the noise parameters and the blur kernel in Wiener deconvolution can be simultaneously optimized with the optical parameters in the lens. Extensive comparative tests have been conducted to demonstrate the single-lens imaging system obtained by our method has more stable imaging quality and a 40 times greater imaging speed than the method using CNN restoration algorithm.