Forecasting mortality is challenging. In general, mortality rate forecasting exercises have been based on the supposition that predictors residuals are random noise. However, issues regarding model selection, misspecification, or the dynamic behavior of the temporal phenomenon lead to biased or underperformed single models. Residual series might present temporal patterns that can still be used to improve the forecasting system. This paper proposes a new multi-step Hybrid System for Mortality Forecasting (HyS-MF) that combines ARIMA with N-BEATS. HyS-MF employs (i) ARIMA to model the mortality time series and (ii) one expert N-BEATS to forecast the residuals in each specific time horizon. The final output is generated by summing ARIMA with the N-BEATS forecasts in each time horizon. HyS-MF achieved a Mean Absolute Percentage Error (MAPE) average less than 1.6% considering all prediction horizons, beating statistical techniques, machine learning models, and hybrid systems considering 20 different time series from the French population mortality rate.