Conductor moving in magnetic field is quite common in electrical equipment. The numerical simulation of such problem is vital in their design and analysis of electrical equipment. The Galerkin finite element method (GFEM) is a commonly employed simulation tool, nonetheless, due to its inherent numerical instability at higher velocities, the GFEM requires upwinding techniques to handle moving conductor problems. The Streamline Upwinding/Petrov-Galerkin (SU/PG) scheme is a widely acknowledged upwinding technique, despite its error-peaking at the transverse boundary. This error at the transverse-boundary, is found to be leading to non-physical solutions. Several remedies have been suggested in the allied fluid dynamics literature, which employs non-linear, iterative techniques. The present work attempts to address this issue, by retaining the computational efficiency of the GFEM. By suitable analysis, it is shown that the source of the problem can be attributed to the Coulomb's gauge. Therefore, to solve the problem, the Coulomb's gauge is taken out from the formulation and the associated weak form is derived. The effectiveness of this technique is demonstrated with pertinent numerical results.