Electrochemical impedance spectroscopy (EIS) is a characterization technique widely used to evaluate the properties of electrochemical systems. The distribution of relaxation times (DRT) has emerged as a model-free alternative to equivalent circuits and physical models to circumvent the inherent challenges of EIS analysis. Deep neural networks (DNNs) can be used to deconvolve DRTs, but several issues remain, e.g., the long training time, the DNN accuracy, and the deconvolution of DRTs with negative peaks. The DNN-DRT model was developed here to address these fundamental limitations. Specifically, a pretraining step was included to decrease the computation time. A thorough error analysis was also conducted to evaluate the different components of the DRT and impedance errors to ultimately decrease them. Lastly, the training loss function was modified to handle DRTs with negative peaks. These different advances were validated with an array of synthetic EIS spectra and real EIS spectra from a lithium-ion battery, a solid oxide fuel cell, and a proton exchange membrane fuel cell. Moreover, this new model outperformed in most cases the previously developed DRTtools and deep-DRT model. Overall, we envision that this research will open the venue for more DNN-based analyses of EIS data for electrochemical systems.