This paper proposes a detumbling motion planning algorithm for free-flying space manipulator with a grasped tumbling target in the post-capturing phase. This algorithm can not only collision-freely guide the space manipulator and the target to terminal stationary states but also suppress the residual vibration of the flexible appendage on the space manipulator. First, considering the avoidances of self-collisions and motion singularities, a smooth detumbling path is planned for the space manipulator by a proposed smoothing rapid random tree star algorithm (SM-RRT*). Second, a quintic polynomial function is implemented to generate a continuous detumbling trajectory along the detumbling path. Then, with the object of minimizing the residual flexible vibrations and the constrains of joint acceleration limits, an optimization model is established to refine the detumbling trajectory. Finally, the optimization model is solved by an improved particle swarm optimization algorithm (PSO), where a potential field term is included in the generation of the particle velocity to enhance the computational efficiency. Simulation results validate the effectiveness of the proposed detumbling motion planning algorithm.