The human–computer interaction attribute of the interactive genetic algorithm (IGA) allows users to participate in the product design process for which the product needs to be evaluated, and requiring a large number of evaluations would lead to user fatigue. To address this issue, this paper utilizes an XGBoost proxy model modified by particle swarm optimization and the graphical interaction mechanism (GIM) to construct an improved interactive genetic algorithm (PXG-IGA), and then the PXG-IGA is applied to 3D vase design. Firstly, the 3D vase shape has been designed by using a bicubic Bézier surface, and the individual genetic code is binary and includes three parts: the vase control points, the vase height, and the texture picture. Secondly, the XGBoost evaluation of the proxy model has been constructed by collecting user online evaluation data, and the particle swarm optimization algorithm has been used to optimize the hyperparameters of XGBoost. Finally, the GIM has been introduced after several generations, allowing users to change product styles independently to better meet users’ expectations. Based on the PXG-IGA, an online 3D vase design platform has been developed and compared to the traditional IGA, KD tree, random forest, and standard XGBoost proxy models. Compared with the traditional IGA, the number of evaluations has been reduced by 58.3% and the evaluation time has been reduced by 46.4%. Compared with other proxy models, the accuracy of predictions has been improved up from 1.3% to 20.2%. To a certain extent, the PXG-IGA reduces users’ operation fatigue and provides new ideas for improving user experience and product design efficiency.