The introduction of deep learning (DL) models and data analysis may significantly elevate the performance of traffic speed prediction. Adverse weather causes mobility and safety concerns because of varying traffic speeds with poor visibility and road conditions. Most previous modeling approaches have not considered the heterogeneity of temporal and spatial data, such as traffic and weather conditions. This paper presents a framework, consisting of two DL models, to predict traffic speed under normal conditions and during adverse weather, considering prevailing traffic speed, wind speed, traffic volume, road capacity, wind bearing, precipitation intensity, and visibility. To ensure the accuracy of speed prediction, different DL models were assessed. The results indicated that the proposed one-dimensional convolutional neural network model outperformed others in relation to the least root mean square error and the least mean absolute error. Considering real-time weather data feeds on a 15-min basis, a tool was also developed for displaying predicted traffic speeds on New Jersey freeways. Application of the proposed framework models for predicting spatio-temporal hot-spot congestion caused by adverse weather is discussed.