Animal-vehicle collision (AVC) is a major concern in road safety that affects human life, properties, and wildlife. Most of the collisions happen with large animals especially deer that enters the road suddenly. Furthermore, the threat is even more alarming in poor visibility conditions such as night-time, fog, rain, etc. Therefore, it is vital to detect the presence of deer on roadways to mitigate the severity of deer-vehicle collision (DVC). This paper presents an efficient methodology to detect deer on roadways both during the day and night-time conditions using deep learning framework. A two-class CNN model differentiating a deer from its background is developed. The background will have a few classes of objects such as motorcycles, cars, and trees which are frequently encountered on roadways. A selfconstructed dataset with both RGB and thermal images is used to train the CNN model. Sliding window technique is used to localize the spatial region of deer in an image. The performance of the proposed CNN model is compared with state-of-the art classifiers and pre-trained CNN models and the results validate its effectiveness.