In this paper, we describes the simultaneous perturbation particle swarm optimization which is a combination of the particle swarm optimization and the simultaneous perturbation optimization method. The method has global search capability of the particle swarm optimization and local search one of gradient method by the simultaneous perturbation. Some variations of the method are described. Comparison between these methods and the ordinary particle swarm optimization are shown through five test functions and learning problem of neural networks.