In view of the difficulties of fruit cluster identification, the specific harvesting sequence constraints of aggregated fruits, and the balanced harvesting task assignment for the multiple arms with a series-increasing symmetric shared (SISS) region, this paper proposes a multi-objective optimization algorithm, which combines genetic algorithm (GA) and ant colony optimization (ACO) stepwise, to optimize the multiarm cooperative harvesting trajectory of straw-rotting fungus to effectively improve the harvesting efficiency and the success rate of non-destructive harvesting. In this approach, firstly, the multiarm trajectory optimization problem is abstracted as a multiple travelling salesman problem (MTSP). Secondly, an improved local density clustering algorithm is designed to identify the cluster fruits to prepare data for harvesting aggregated fruits in a specific order later. Thirdly, the MTSP has been decomposed into M independent TSP (traveling salesman problem) problems by using GA, in which a new DNA (deoxyribonucleic acid) assignment rule is designed to resolve the problem of the average distribution of multiarm harvesting tasks with the SISS region. Then, the improved ant colony algorithm, combined with the auction mechanism, is adopted to achieve the shortest trajectory of each arm, which settles the difficulty that the clustered mature fruits should be harvested in a specified order. The experiments show that it can search for a relatively stable optimal solution in a relatively short time. The average harvesting efficiency is up to 1183 pcs/h and the average harvesting success rate is about 97%. Therefore, the proposed algorithm can better plan the harvesting trajectory for multiarm intelligent harvesting, especially for areas with many aggregated fruits.