In this paper, an ammonia decomposition membrane reactor is applied to a solar heat absorption system, and thermodynamic optimization is carried out according to the usage scenarios. First, a model of an ammonia decomposition solar heat absorption system based on the membrane reactor is established by using finite time thermodynamics (FTT) theory. Then, the three-objective optimization with and the four-objective optimization without the constraint of the given heat absorption rate are carried out by using the NSGA-II algorithm. Finally, the optimized performance objectives and the corresponding design parameters are obtained by using the TOPSIS decision method. Compared with the reference system, the TOPSIS optimal solution for the three-objective optimization can reduce the entropy generation rate by 4.8% and increase the thermal efficiency and energy conversion rate by 1.5% and 1.4%, respectively. The optimal solution for the four-objective optimization can reduce the heat absorption rate, entropy generation rate, and energy conversion rate by 15.5%, 14%, and 8.7%, respectively, and improve the thermal efficiency by 15.7%. The results of this paper are useful for the theoretical study and engineering application of ammonia solar heat absorption systems based on membrane reactors.