Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting

Lin, Chen; Ouyang, Zhichao; Zhuang, Junqing; Chen, Jianqiang; Li, Hui; Wu, Rongxin

doi:10.1109/icpc52881.2021.00026

Cited by 47 publications

(18 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An important but underexplored direction is multi-lingual commonsense reasoning and generation. Studies have shown that the performance of cross-lingual PLMs is poor when evaluated on non-English commonsense reasoning benchmarks (Lin et al 2021b). These models perform poorly when evaluated on a test set that was translated to English, leading to staggering transfer reasoning capabilities to other languages and restricting the research scope to only certain languages (Ponti et al 2020).…”

Section: Discussionmentioning

confidence: 99%

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Bhargava¹,

Ng²

2022

AAAI

View full text Add to dashboard Cite

While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks. This paper presents a survey of these tasks, discusses the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation as revealed by these tasks, and reflects on future research directions.

show abstract

Section: Discussionmentioning

confidence: 99%

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Bhargava¹,

Ng²

2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Jiang et al [23] represent ASTs as the set of paths and then introduce node position embedding to obtain the position of the node in the AST. Besides, neural networks that take trees as input (e.g., tree-LSTMs [33,53,54], RvNNs [61] and GNNs/GCNs [9,29,59]) utilize an AST directly instead of flattening it.…”

Section: Structural Information Of Source Codementioning

confidence: 99%

SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations

Niu¹,

Li²,

Ng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent years have seen the successful application of large pretrained models to code representation learning, resulting in substantial improvements on many code-related downstream tasks. But there are issues surrounding their application to SE tasks. First, the majority of the pre-trained models focus on pre-training only the encoder of the Transformer. For generation tasks that are addressed using models with the encoder-decoder architecture, however, there is no reason why the decoder should be left out during pre-training. Second, many existing pre-trained models, including state-of-the-art models such as T5-learning, simply reuse the pretraining tasks designed for natural languages. Moreover, to learn the natural language description of source code needed eventually for code-related tasks such as code summarization, existing pretraining tasks require a bilingual corpus composed of source code and the associated natural language description, which severely limits the amount of data for pre-training. To this end, we propose SPT-Code, a sequence-to-sequence pre-trained model for source code. In order to pre-train SPT-Code in a sequence-to-sequence manner and address the aforementioned weaknesses associated with existing pre-training tasks, we introduce three pre-training tasks that are specifically designed to enable SPT-Code to learn knowledge of source code, the corresponding code structure, as well as a natural language description of the code without relying on any bilingual corpus, and eventually exploit these three sources of information when it is applied to downstream tasks. Experimental

show abstract

“…Although a flow graph is more likely to be seen as a graph, there are few works that treat flow graphs as trees. For example, BAST [103] splits the code of a method according to the blocks in the dominator tree of CFG and generates the corresponding AST for the split code. The split ASTs' representations are used in the pre-trained stage by predicting the next split AST in the dominator tree.…”

Section: Flow-graph-based Structuresmentioning

confidence: 99%

“…Pre-embeds the interprocedural value-flow graph, considers the reachability via matrix multiplication problem and uses it to approximate the high-order proximity embedding BASTS [103] Tree-LSTM Uses Tree-LSTM and Transformer architecture to combine the representations of split AST and source code CoCoSum [164] Transformer, Multi-Relational GNN…”

Section: Clone Detectionmentioning

confidence: 99%

A Survey of Deep Learning Models for Structural Code Understanding

Wu¹,

Zhang²,

Peng³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, the rise of deep learning and automation requirements in the software industry has elevated Intelligent Software Engineering to new heights. The number of approaches and applications in code understanding is growing, with deep learning techniques being used in many of them to better capture the information in code data. In this survey, we present a comprehensive overview of the structures formed from code data. We categorize the models for understanding code in recent years into two groups: sequence-based and graph-based models, further make a summary and comparison of them 1 . We also introduce metrics, datasets and the downstream tasks. Finally, we make some suggestions for future research in structural code understanding field.

show abstract

Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting

Cited by 47 publications

References 38 publications

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations

A Survey of Deep Learning Models for Structural Code Understanding

Contact Info

Product

Resources

About