Variability in solar irradiance has an impact on the stability of solar systems and the grid’s safety. With the decreasing cost of solar panels and recent advancements in energy conversion technology, precise solar energy forecasting is critical for energy system integration. Despite extensive research, there is still potential for advancement of solar irradiance prediction accuracy, especially global horizontal irradiance. Global Horizontal Irradiance (GHI) (unit: KWh/m2) and the Plane Of Array (POA) irradiance (unit: W/m2) were used as the forecasting objectives in this research, and a hybrid short-term solar irradiance prediction model called modified multi-step Convolutional Neural Network (CNN)-stacked Long-Short-Term-Memory network (LSTM) with drop-out was proposed. The real solar data from Sweihan Photovoltaic Independent Power Project in Abu Dhabi, UAE is preprocessed, and features were extracted using modified CNN layers. The output result from CNN is used to predict the targets using a stacked LSTM network and the efficiency is proved by comparing statistical performance measures in terms of Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Squared Error (MAE), and R2 scores, with other contemporary machine learning and deep-learning-based models. The proposed model offered the best RMSE and R2 values of 0.36 and 0.98 for solar irradiance prediction and 61.24 with R2 0.96 for POA prediction, which also showed better performance as compared to the published works in the literature.