Mathematical Expressions in Software Engineering Artifacts

Sur, Tanuj; Jaisswal, Aaditree; Vinayakarao, Venkatesh

doi:10.1145/3570991.3571033

Cited by 2 publications

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

TEIMMA: The First Content Reuse Annotator for Text, Images, and Math

Satpute,

Greiner-Petter,

Schubotz

et al. 2023

2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

View full text Add to dashboard Cite

Large Language Models (LLMs) have demonstrated exceptional capabilities in various natural language tasks, often achieving performances that surpass those of humans. Despite these advancements, the domain of mathematics presents a distinctive challenge, primarily due to its specialized structure and the precision it demands. In this study, we adopted a two-step approach for investigating the proficiency of LLMs in answering mathematical questions. First, we employ the most effective LLMs, as identified by their performance on math question-answer benchmarks, to generate answers to 78 questions from the Math Stack Exchange (MSE). Second, a case analysis is conducted on the LLM that showed the highest performance, focusing on the quality and accuracy of its answers through manual evaluation. We found that GPT-4 performs best (nDCG of 0.48 and P@10 of 0.37) amongst existing LLMs finetuned for answering mathematics questions and outperforms the current best approach on ArqMATH3 Task1, considering P@10. Our Case analysis indicates that while the GPT-4 can generate relevant responses in certain instances, it does not consistently answer all questions accurately. This paper explores the current limitations of LLMs in navigating complex mathematical problemsolving. Through case analysis, we shed light on the gaps in LLM capabilities within mathematics, thereby setting the stage for future research and advancements in AI-driven mathematical reasoning. We make our code and findings publicly available for research: https://github.com/gipplab/LLM-Investig-MathStackExchange CCS CONCEPTS• Computing methodologies → Discourse, dialogue and pragmatics; • Mathematics of computing → Mathematical software performance.

show abstract

TEIMMA: The First Content Reuse Annotator for Text, Images, and Math

Satpute,

Greiner-Petter,

Schubotz

et al. 2023

2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

View full text Add to dashboard Cite

show abstract

Requirement Change Prediction Model for Small Software Systems

et al. 2023

View full text Add to dashboard Cite

The software industry plays a vital role in driving technological advancements. Software projects are complex and consist of many components, so change is unavoidable in these projects. The change in software requirements must be predicted early to preserve resources, since it can lead to project failures. This work focuses on small-scale software systems in which requirements are changed gradually. The work provides a probabilistic prediction model, which predicts the probability of changes in software requirement specifications. The first part of the work considers analyzing the changes in software requirements due to certain variables with the help of stakeholders, developers, and experts by the questionnaire method. Then, the proposed model incorporates their knowledge in the Bayesian network as conditional probabilities of independent and dependent variables. The proposed approach utilizes the variable elimination method to obtain the posterior probability of the revisions in the software requirement document. The model was evaluated by sensitivity analysis and comparison methods. For a given dataset, the proposed model computed the low state revisions probability to 0.42, and the high state revisions probability to 0.45. Thus, the results proved that the proposed approach can predict the change in the requirements document accurately by outperforming existing models.

show abstract

Mathematical Expressions in Software Engineering Artifacts

Cited by 2 publications

References 19 publications

TEIMMA: The First Content Reuse Annotator for Text, Images, and Math

TEIMMA: The First Content Reuse Annotator for Text, Images, and Math

Requirement Change Prediction Model for Small Software Systems

Contact Info

Product

Resources

About