This study investigated the synthesis of α-hemihydrate gypsum (α-HH) through semi-liquid autoclaving of phosphogypsum (PG) using various carboxylic acids as modifying agents. The impact of carboxyl group spatial location, auxiliary functional group type, and the number of carboxyl groups within the carboxylic acid modifiers on the mechanical strength and crystal morphology of α-HH was analyzed using scanning electron microscopy (SEM), strength testing, and molecular dynamics simulations. The results revealed a significant influence of the carboxylic acid molecular structure on the α-HH crystal morphology. Monocarboxylic acids and dicarboxylic acids with a long carbon chain length between carboxyl groups exhibited preferential adsorption on the (200), (110), and (−110) crystal planes, promoting crystal growth along the c-axis. In contrast, hydroxyl groups and cis double bonds in the modifier structure induced selective adsorption on the (001) plane, hindering growth along the c-axis. Conversely, trans double bonds favored adsorption on the (200), (110), and (−110) planes, enhancing growth along the c-axis. Based on these observations, screening principles for carboxylic acid modifiers were established, suggesting that: 1) the number of carboxyl groups should exceed 2; 2) the optimal carbon atom spacing between carboxyl groups is 3; and 3) auxiliary functional groups such as hydroxyl groups and cis double bonds should be introduced. Modifiers like citric acid, ethylene diamine tetraacetic acid (EDTA), and pyromellitic acid, within concentration ranges of 0.05%–0.1%, 0.1%–0.15%, and 0.05%–0.1%, respectively, yielded α-HH with flexural strengths exceeding 4 MPa and compressive strengths greater than 35 MPa, demonstrating the validity of these principles.