Next generation networks are envisioned to be heterogeneous in nature with an increase in demand towards ubiquitous services in wireless networks. As various networks have widely different characteristics, it is difficult to maintain the quality of service (QoS) after executing a handoff from one network to another network. Maintaining the QoS, based on applications, during the handoff in an heterogeneous networks needs an intelligent handoff decision mechanism. This article proposes a QoS-aware fuzzy rule-based vertical handoff mechanism that makes a multi-criteria-based decision, found to be effective for meeting the requirements of different applications in a heterogeneous networking environment. The QoS parameters considered are available bandwidth, end-to-end delay, jitter, and bit error rate. A new evaluation model is proposed using a non-birth-death Markov chain, in which the states correspond to the available networks. Simulation results show that compared to other vertical handoff algorithms, the proposed algorithm gives better performance for different traffic classes.