Heavy quarks are valuable probes of the electromagnetic field and the initial condition of the quark-gluon plasma (QGP) matter produced in high-energy nuclear collisions. Within an improved Langevin model that is coupled to a (3+1)-dimensional viscous hydrodynamic model, we explore the origin of the directed flow coefficient (v1) of heavy mesons and their decay leptons, and its splitting (∆v1) between opposite charges. We find that while the rapidity dependence of the heavy flavor v1 is mainly driven by the titled energy density profile of the QGP with respect to the longitudinal direction at the RHIC energy, it is dominated by the electromagnetic field at the LHC energy. The ∆v1 serves as a novel probe of the spacetime evolution profile of the electromagnetic field. Our results of D mesons and their decay electrons are consistent with the available data at RHIC and LHC, and our predictions on the heavy flavor decay muons can be further tested by future measurements.