Wireless technologies have evolved very rapidly in recent years. In the future, operators will need to enable users to use communication services independently of access technologies, so they will have to support seamless handovers in heterogeneous networks. In this paper we present a novel adaptive congestion aware Session Initiation Protocol (SIP) based procedure for handover in heterogeneous networks. In the proposed algorithm the handover decision is based in addition to signal strength, also on target network congestion status, which is tested during the conversation. As SIP protocol was used, the proposed procedure is independent of access technologies. For performance evaluation of proposed procedure we developed a purpose built simulation model. The results show that the use of the proposed adaptive procedure significantly improves the QoE of VoIP users, compared to reference scenario, in which only signal strength was used as the trigger for handover decision.