The penetration of PVs into the power grid is increasing day by day, as they are more economical and environment-friendly. However, due to the intrinsic intermittency in solar radiation and other meteorological factors, the generated power from PVs is uncertain and unstable. Therefore, accurate forecasting of power generation is considered one of the fundamental challenges in power system. In this paper, a deep-learning model based on two-stage attention mechanism over LSTM is proposed to forecast a day-ahead PV power. In addition, the Bayesian optimization algorithm is applied to obtain the optimal combination of hyper-parameters for the proposed deep-learning model. Various input features that can affect the PV power generation such as solar radiation, temperature, humidity, snowfall, albedo etc. are considered and their impact with respect to the attention mechanisms on the forecasted PV power is analyzed. The input consists of data from 21 PVs installed at different geographical locations in Germany. The proposed model is compared with state-of-the-art models such as LSTM-Attention, CNN-LSTM, and Ensemble model for day-ahead forecasting. The model is also compared with various single attention mechanisms such as Inputattention, SNAIL, Raffel, and Hierarchical attention etc. The proposed model outperforms the traditional methods in terms of accuracy, hence proving its efficiency. Forecasting Skill (FS) score of the proposed model is 0.4813 whereas 0.4427 is for the Ensemble model, which is the best among other state-of-the-art models. Root Mean Square (RMSE) and Mean Absolute Error (MAE) of the proposed model is 0.0638 and 0.0324 respectively, whereas those of the Ensemble model are 0.0685 and 0.0369 respectively.