Augmented reality (AR) technology provides context-aware experiences by overlaying digital information onto the real world to enhance learning effectiveness and reduce cognitive load. This study aimed to develop an AR Mobile Learning System (ARMLS) to address the limitations of traditional teaching materials and help elementary-school students learn geometric concepts. The ARMLS was designed based on the fifth-grade mathematics curriculum, covering topics such as definitions, geometric properties, different views of prisms and pyramids, and their relationships. A teaching experiment was conducted to compare students’ learning achievement, motivation, and cognitive load when using the ARMLS versus traditional teaching materials. This study adopted a quasi-experimental design, where four fifth-grade classes were selected from an elementary school in northern Taiwan as experimental subjects. A total of 66 students participated in the experiment, divided into two groups: 32 students from two classes as the experimental group (using the ARMLS) and 34 students from the other two classes as the control group (using traditional teaching materials). In the teaching experiment, data were collected through pre-tests, post-tests, and questionnaires. Achievement tests assessed learning effectiveness, while learning motivation and cognitive load were measured with standardized scales. System satisfaction was evaluated using a questionnaire. The Johnson–Neyman method determined the regions of significance in the analysis of covariance. Independent-sample t-tests evaluated differences in learning motivation and cognitive load between the groups, and descriptive statistics summarized system satisfaction responses. The results indicated that (1) the ARMLS enhanced the learning achievement among low- and moderate-achieving students, (2) there was no significant difference in learning motivation between the two groups, (3) the ARMLS helped reduce students’ cognitive load, and (4) most students expressed satisfaction with the ARMLS according to the questionnaire results. The ARMLS enhances engagement and deepens understanding by making abstract geometry topics more accessible. It effectively overcomes the limitations of traditional teaching materials, providing elementary students with an interactive, hands-on approach to learning geometric concepts.