Solving Geometry Problems via Feature Learning and Contrastive Learning of Multimodal Data

Abstract

This paper presents an end-to-end deep learning method to solve geometry problems via feature learning and contrastive learning of multimodal data. A key challenge in solving geometry problems using deep learning is to automatically adapt to the task of understanding single-modal and multimodal problems. Existing methods either focus on single-modal or multimodal problems, and they cannot fit each other. A general geometry problem solver should obviously be able to process various modal problems at the same time. In this paper, a shared feature-learning model of multimodal data is adopted to learn the unified feature representation of text and image, which can solve the heterogeneity issue between multimodal geometry problems. A contrastive learning model of multimodal data enhances the semantic relevance between multimodal features and maps them into a unified semantic space, which can effectively adapt to both single-modal and multimodal downstream tasks. Based on the feature extraction and fusion of multimodal data, a proposed geometry problem solver uses relation extraction, theorem reasoning, and problem solving to present solutions in a readable way. Experimental results show the effectiveness of the method.

Interpretable Geometry Problem Solving Using Improved RetinaNet and Graph Convolutional Network

“…Zhang [18] constructed a new large-scale geometric diagram dataset PGDP5K, proposed an improved instance segmentation method, and combined geometric features and prior knowledge to achieve relationship classification and analysis. Jian et al [19,20] used feature learning and contrastive learning to solve geometry problems. Lu [10] constructed a Geometry3K dataset consisting of 3002 geometric problems annotated with formal language.…”

FGeo-SSS: A Search-Based Symbolic Solver for Human-like Automated Geometric Reasoning

“…Shared benchmarks and datasets have significantly advanced research in AI-assisted GPS. Several AI systems, such as the CL-based model [56], SCA [57], GeoDRL [14], and LANS [58], have been constructed to achieve higher problem-solving success rates.…”