Particle tuned mass damper has received considerable attention on mitigating structural seismic responses recently. However, soil-structure interaction effects have not been considered for the optimization design of particle tuned mass damper in the previous studies yet, which may play significant roles in particle tuned mass damper’s effectiveness. This paper investigates the seismic control performance of particle tuned mass damper for tall buildings considering SSI effects. A 40-story benchmark structure is adopted as the primary structure, and different soil conditions including soft, medium, dense soil, and the fixed-base case are considered. Based on the Cuckoo Search algorithm, an optimum design approach for the particle tuned mass damper implemented in SSI system under earthquake excitations is proposed, and the optimum parameters of particle tuned mass damper are obtained from the optimum design process. The optimum results indicate that the optimized particle tuned mass damper can effectively decrease the displacement response without the increase of peak acceleration, and the reduction rates could exceed 30% in some cases. Compared with other soil type cases, under soft soil condition, the optimum parameters of particle tuned mass damper are especially different, and the mitigation effects of the optimized particle tuned mass damper on maximum displacement weaken. Hence, it is necessary to consider the SSI effects for particle tuned mass damper’s seismic design. Furthermore, the interaction effects between the primary structure and particle tuned mass damper in SSI system are also evaluated through energy analysis. The results show that considering SSI effects, the optimized particle tuned mass damper device dissipates the overwhelming majority of the input energy, and it greatly decreases the structural energy, displaying excellent energy dissipation performance on seismic vibration control of structure.