Q-learning is a reinforcement learning method for solving Markov decision problems with incomplete information proposed by Watkins. With the develop of reinforcement learning, more and more Q-learning related algorithms have been proposed, and their application range has become wider. In this paper, we discussed single agent algorithms including basic Q learning, deep Q learning and double Q learning. In addition, we discussed multi-agent algorithms including modular Q learning, ant Q learning and Nash Q learning with prominent characteristics. This paper will compare their advantages and disadvantages, and put forward our own views on the current application of Q-learning and the future trend of Q-learning.