The coronavirus COVID-19 is affecting 213 countries and territories around the world. Iran was one of the first affected countries by this virus. Isfahan, as the third most populated province of Iran, experienced a noticeable epidemic. The prediction of epidemic size, peak value, and peak time can help policymakers in correct decisions. In this study, deep learning is selected as a powerful tool for forecasting this epidemic in Isfahan. A combination of effective Social Determinant of Health (SDH) and the occurrences of COVID-19 data are used as spatiotemporal input by using time-series information from different locations. Different models are utilized, and the best performance is found to be for a tailored type of long short-term memory (LSTM). This new method incorporates mutual effect of all classes (confirmed/ death / recovered) in predication process. The future trajectory of the outbreak in Isfahan is forecasted with the proposed model. The paper demonstrates the positive effect of adding SDHs in pandemic prediction. Furthermore, the effectiveness of different SDHs is discussed, and the most effective terms are introduced. The method expresses high ability in both short- and long- term forecasting of the outbreak. The model proves that in predicting one class (like the number of confirmed cases), the effect of other accompanying numbers (like death and recovered cases) cannot be ignored. In conclusion, the superiorities of this model (particularity the long term predication ability) turn it into a reliable tool for helping the health decision makers.