IEEE 802.16e specification adopted that a target base station (BS) transmits Fast_Ranging_IE() to the upcoming handover MS to reduce the handover interruption time. However, the handover interruption time can be increased if the MS fails to receive Fast_Ranging_IE() because the channel release time between the MS and the serving BS is increased more than the estimated handover time due to the queuing delay and the handover message loss. In this paper, we propose the handover time negotiation mechanism to receive Fast_Ranging_IE() exactly. The proposed mechanism helps the serving BS and neighbor BSs to negotiate and share the estimated handover time information each other. To determine the estimated handover time, we considers the queuing delay in wireless and wired network, and the delay due to the handover message loss. The simulation results show that the receiving failure ratio of Fast_Ranging_IE() is reduced about 94% and handover interruption time of 95% users is less than 150msec when the proposed mechanism is applied to IEEE 802.16e system. Therefore, the proposed mechanism helps the IEEE 802.16e system to support the seamless service for the real-time applications.