Rainfall forecasting, especially extreme rainfall forecasting, is one of crucial tasks in weather forecasting since it has direct impact on accompanying devastating events such as flash floods and fast-moving landslides. However, obtaining rainfall forecasts with high accuracy, especially for extreme rainfall occurrences, is a challenging task. This study focuses on developing a forecasting model which is capable of forecasting rainfall, including extreme rainfall values. The rainfall forecasting was achieved through sequence learning capability of the Long Short-Term Memory (LSTM) method. The identification of the optimal set of features for the LSTM model was conducted using Random Forest and Granger Causality tests. Then, that best set of features was fed into Stacked LSTM, Bidirectional LSTM, and Encoder-Decoder LSTM models to obtain three days-ahead forecasts of rainfall with the input of the past fourteen days-values of selected features. Out of the three models, the best model was taken through post hoc residual analysis and extra validation approaches. This entire approach was illustrated utilizing rainfall and weather-related measurements obtained from the gauging station located in the city of Ratnapura, Sri Lanka. Originally, twenty-three features were collected including relative humidity, ssunshine hours, and mean sea level pressure. The performances of the three models were compared using RMSE. The Bidirectional LSTM model outperformed the other methods (RMSE < 5 mm and MAE < 3 mm) and this model has the capability to forecast extreme rainfall values with high accuracy.