To forecast the long-term ageing deviations of capacitors, a new highly-performing deep neural network (DNN) architecture is proposed based upon the long short-term memory (LSTM) algorithm. By importing the early ageing data with respect to the applied thermal and electrical stresses into the proposed LSTM-based DNN architecture, the future accelerated ageing-induced deviations in capacitance and ESR (Equivalent Series Resistance) are predicted accordingly. The dropout and the prediction interval technique are applied to overcome overfitting issues and obtain an uncertainty estimation. The results indicate that the proposed LSTM-based DNN algorithm has a higher prediction accuracy and narrower prediction intervals compared with other deep learning methods.