In recent years, the emerging paradigm of software-defined networking has become a hot and thriving topic that grabbed the attention of industry sector as well as the academic research community. The decoupling between the network control and data planes means that software-defined networking architecture is programmable, adjustable and dynamically re-configurable. As a result, a large number of leading companies across the world have latterly launched software-defined solutions in their data centers and it is expected that most of the service providers will do so in the near future due to the new opportunities enabled by software-defined architectures. Nonetheless, each emerging technology is accompanied by new issues and concerns, and fault tolerance and recovery is one such issue that faces software-defined networking. Although there have been numerous studies that have discussed this issue, gaps still exist and need to be highlighted. In this paper, we start by tracing the evolution of networking systems from the mid 1990's until the emergence of programmable networks and software-defined networking, and then define a taxonomy for software-defined networking dependability by means of fault tolerance of data plane to cover all aspects, challenges and factors that need to be considered in future solutions. We discuss in a detailed manner current state-of-the-art literature in this area. Finally, we analyse the current gaps in current research and propose possible directions for future work.