In a recent article, Kwon et al. [Nature (London) 600, 64 (2021)] revealed non-universal dissipative dynamics of quantum vortices in a fermionic superfluid. The enhancement of the dissipative process is pronounced for Bardeen-Cooper-Schrieffer interaction regime, and it was suggested that the effect is due to the presence of quasiparticles localized inside the vortex core. We test this hypothesis through numerical simulations with time-dependent density functional theory: a fully microscopic framework with fermionic degrees of freedom. The results of fully microscopic calculations expose the impact of the vortex-bound states on dissipative dynamics in a fermionic superfluid. Their contribution is too weak to explain the experimental measurements, and we identify that thermal effects, giving rise to mutual friction between superfluid and the normal component, dominate the observed dynamics.