The operation conditions of series transformer in Unified Power Flow Controller (UPFC) system are analyzed, and a shell-type structural scheme is proposed in this paper. Taking into account the nonlinear characteristic and the eddy current effect of the ferromagnetic materials, the effective complex permeability is adopted to calculate the nonlinear reluctance of the core, and an electromagnetic transient model of the series transformer is presented based on the duality theory of electric circuit and magnetic circuit, which is one key part of the comprehensive simulation model of UPFC. Besides, the improved magnetic circuit method is coupled with the electric circuit to analyze the special three-phase asymmetrical DC bias phenomenon of series transformer. Some experiments about no-load switching and asymmetrical DC bias on a low-voltage series transformer prototype have been done to verify the transient model and the improved magnetic circuit method, respectively. Finally, the reactive power of the no-load series transformer prototype under asymmetrical DC bias is calculated, the results indicate that the variation of the reactive power of series transformer under asymmetrical DC bias becomes more complicated, which increases the difficulty of voltage regulation in UPFC system.