Effective Approach for Fine-Tuning Pre-Trained Models for the Extraction of Texts From Source Codes

Abstract

This study introduces SR-Text, a robust approach leveraging pre-trained models like BERT and T5 for enhanced text extraction from source codes. Addressing the limitations of traditional manual summarization, our methodology focuses on fine-tuning these models to better understand and generate contextual summaries, thus overcoming challenges such as long-term dependency and dataset quality issues. We conduct a detailed analysis of programming language syntax and semantics to develop syntax-aware text retrieval techniques, significantly boosting the accuracy and relevance of the texts extracted. The paper also explores a hybrid approach that integrates statistical machine learning with rule-based methods, enhancing the robustness and adaptability of our text extraction processes across diverse coding styles and languages. Empirical results from a meticulously curated dataset demonstrate marked improvements in performance metrics: precision increased by 15%, recall by 20%, and an F1 score enhancement of 18%. These improvements underscore the effectiveness of using advanced machine learning models in software engineering tasks. This research not only paves the way for future work in multilingual code summarization but also discusses broader implications for automated software analysis tools, proposing directions for future research to further refine and expand this methodology.