Hybrid classical quantum optimization methods have become an important tool for efficiently solving problems in the current generation of NISQ computers. These methods use an optimization algorithm that is executed in a classical computer, which is fed with values of the objective function that are obtained in a quantum computer. Therefore, a proper choice of optimization algorithm is essential to achieve good performance. Here, we review the use of first-order, second-order, and quantum natural gradient stochastic optimization methods, which are defined in the field of real numbers, and propose new stochastic algorithms defined in the field of complex numbers. The performance of the methods is evaluated by means of their application to variational quantum eigensolver, quantum control of quantum states, and quantum state estimation. In general, complex number optimization algorithms perform better and second-order algorithms provide a large improvement over first-order methods in the case of variational quantum eigensolver.