EEG source localization remains a challenging problem given the uncertain conductivity values of the volume conductor models (VCMs). As uncertain conductivities vary across people, they may considerably impact the forward and inverse solutions of the EEG, leading to an increase in localization mistakes and misdiagnoses of brain disorders. Calibration of conductivity values using uncertainty quantification (UQ) techniques is a promising approach to reduce localization errors. The widely-known UQ methods involve Bayesian approaches, which utilize prior conductivity values to derive their posterior inference and estimate their optimal calibration. However, these approaches have two significant drawbacks: solving for posterior inference is intractable, and choosing inappropriate priors may lead to increased localization mistakes. This study used the Neural Stochastic Differential Equations Network (SDE-Net), a combination of dynamical systems and deep learning techniques that utilizes the Wiener process to minimize conductivity uncertainties in the VCM and improve the inverse problem. Results revealed that SDE-Net generated a lower localization error rate in the inverse problem compared to Bayesian techniques. Future studies may employ new stochastic dynamical systems-based techniques as a UQ technique to address further uncertainties in the EEG Source Localization problem. Our code can be found here: https://github.com/rrwabina/SDE-Net-UQ-ESL.