When investing in new stocks, it is difficult to predict returns and risks in a general way without the support of historical data. Therefore, a portfolio optimization model with an uncertain rate of return is proposed. On this basis, prospect theory is used for reference, and then the uncertain return portfolio optimization model is established from the perspective of expected utility maximization. An improved gray wolf optimization (GWO) algorithm is designed because of the complex nonsmooth and nonconcave characteristics of the model. The results show that the GWO algorithm is superior to the traditional particle swarm optimization algorithm and genetic algorithm.