EnCoSum: enhanced semantic features for multi-scale multi-modal source code summarization

Gao, Yuexiu; Zhang, Hongyu; Lyu, Chen

doi:10.1007/s10664-023-10384-x

Cited by 3 publications

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• CodeT5 [16] is a pre-trained encoder-decoder model based on T5 [32] for programming and natural language, directly tuned with summarization datasets. • CodeBERT [10] is a pre-trained encoder model for programming and natural language based on Roberta [33].…”

Section: B Baselinementioning

confidence: 99%

GraphPLBART: Code Summarization Based on Graph Embedding and Pre-Trained Model

Li,

Zhu

et al. 2023

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Code summarization is a task that aims at automatically producing descriptions of source code. Recently many deep-learning-based approaches have been proposed to generate accurate code summaries, among which pre-trained models for programming languages have achieved promising results. It is well-known that source code written in programming languages is highly structured and unambiguous. Though previous work pre-trained the model with well-design tasks to learn universal representation from a large scale of data, they haven't considered structure information during the fine-tuning stage. To make full use of both the pre-trained programming language model and the structure information of source code, we utilize Flow-Augmented Abstract Syntax Tree (FA-AST) of source code for structure information and propose GraphPLBART -Graphaugmented Programming Language and Bi-directional Auto-Regressive Transformer, which can effectively introduce structure information to a well pre-trained model through a cross attention layer. Experimental results show that our approach outperforms the baseline models in some metrics.

show abstract

Section: B Baselinementioning

confidence: 99%

GraphPLBART: Code Summarization Based on Graph Embedding and Pre-Trained Model

Li,

Zhu

et al. 2023

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

Semantic similarity loss for neural source code summarization

Su,

McMillan

2024

J Software Evolu Process

View full text Add to dashboard Cite

This paper presents a procedure for and evaluation of using a semantic similarity metric as a loss function for neural source code summarization. Code summarization is the task of writing natural language descriptions of source code. Neural code summarization refers to automated techniques for generating these descriptions using neural networks. Almost all current approaches involve neural networks as either standalone models or as part of a pretrained large language models, for example, GPT, Codex, and LLaMA. Yet almost all also use a categorical cross‐entropy (CCE) loss function for network optimization. Two problems with CCE are that (1) it computes loss over each word prediction one‐at‐a‐time, rather than evaluating a whole sentence, and (2) it requires a perfect prediction, leaving no room for partial credit for synonyms. In this paper, we extend our previous work on semantic similarity metrics to show a procedure for using semantic similarity as a loss function to alleviate this problem, and we evaluate this procedure in several settings in both metrics‐driven and human studies. In essence, we propose to use a semantic similarity metric to calculate loss over the whole output sentence prediction per training batch, rather than just loss for each word. We also propose to combine our loss with CCE for each word, which streamlines the training process compared to baselines. We evaluate our approach over several baselines and report improvement in the vast majority of conditions.

show abstract