With the rapid development of communication technology and the popularization of network, information security has been highly valued by all walks of life. Random numbers are used in many cryptographic protocols, key management, identity authentication, image encryption, and so on. True random numbers (TRNs) have better randomness and unpredictability in encryption and key than pseudorandom numbers (PRNs). Chaos has good features of sensitive dependence on initial conditions, randomness, periodicity, and reproduction. These demands coincide with the rise of TRNs generating approaches in chaos field. This survey paper intends to provide a systematic review of true random number generators (TRNGs) based on chaos. Firstly, the two kinds of popular chaotic systems for generating TRNs based on chaos, including continuous time chaotic system and discrete time chaotic system are introduced. The main approaches and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review TRNGs based on current-mode chaos for this problem. Finally, quantitative results are given for the described methods in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of TRNGs based on chaos.