The development of graphene oxide/rare earth complexes luminescent materials is limited due to the strong quenching ability of graphene oxide. However, the hybrid materials with pyromellitic acid (PMA) as ligand not only have better performance, but also have stronger luminescence intensity. Herein we tested the effect of the carboxylic acid structure of PMA on the luminescence intensity of graphene oxide/rare earth complexes hybrid materials. Phthalic acid (PA), Isophthalic acid (i-PA) and Terephthalic acid (p-PA) are selected as test ligand, representing the o-position, meta-position and para-position effects of PMA. By comparing the luminescence intensity, we found that o-position effect is helpful for the ligand to sensitize rare earth ions, and meta-position and para-position effects are helpful for the graphene oxide sheets (GOSs) to improve the luminescence intensity of hybrid materials and inhibit the quenching effect of the GOSs. Furthermore, we speculate on the relationship between the distance of π-π stacking of GOSs and rare earth complexes and luminescence intensity of hybrid materials. This work provides an important reference for the design of graphene oxide/rare-earth aromatic carboxylic acid hybrid luminescent materials.