When we take a picture through glass windows, the photographs are often degraded by undesired reflections. To separate reflection layer and background layer is an important problem for enhancing image quality. However, single-image reflection removal is a challenging process because of the ill-posed nature of the problem. In this paper, we propose a single-image reflection removal method based on generative adversarial networks. Our network is an end-to-end trained network with four types of losses. It includes pixel loss, feature loss, adversarial loss, and gradient constraint loss. We propose a novel gradient constraint loss in order to separate the background layer and the reflection layer clearly. Gradient constraint loss is applied in a gradient domain and it minimizes the correlation between the background and reflection layer. Owing to the novel loss and our new synthetic dataset, our reflection removal method outperforms state-of-the-art methods in PSNR and SSIM, especially in real world images.