A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts

Abstract

Code comment has been an important part of computer programs, greatly facilitating the understanding and maintenance of source code. However, high-quality code comments are often unavailable in smart contracts, the increasingly popular programs that run on the blockchain. In this paper, we propose a Multi-Modal Transformer-based (MMTrans) code summarization approach for smart contracts. Specifically, the MMTrans learns the representation of source code from the two heterogeneous modalities of the Abstract Syntax Tree (AST), i.e., Structure-based Traversal (SBT) sequences and graphs. The SBT sequence provides the global semantic information of AST, while the graph convolution focuses on the local details. The MMTrans uses two encoders to extract both global and local semantic information from the two modalities respectively, and then uses a joint decoder to generate code comments. Both the encoders and the decoder employ the multi-head attention structure of the Transformer to enhance the ability to capture the long-range dependencies between code tokens. We build a dataset with over 300K <method, comment> pairs of smart contracts, and evaluate the MMTrans on it. The experimental results demonstrate that the MMTrans outperforms the state-of-the-art baselines in terms of four evaluation metrics by a substantial margin, and can generate higher quality comments.

Cross-Domain Deep Code Search with Few-Shot Meta Learning

“…Used for Fine-tuning and Code Search. We fine-tune and test the code search task using two domain-specific languages, namely, SQL and Solidity [36]. The statistics about the datasets are shown in Table 2.…”

“…Solidity is an object-oriented language that is specifically designed for implementing smart contracts [36]. The dataset of Solidity used in our experiments is provided by [36] for smart contract code summarization. We preprocess the dataset by removing all inline comments from functions.…”

“…They rely heavily on the availability of large parallel data to learn the semantic mappings between code and natural language [12]. On the other hand, there is an emerging trend of domain-specific languages such as Solidity for smart contracts [35,36,41] where code search is also needed. There is often insufficient training data in specific domains, causing poor fitting of deep learning models.…”

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

“…The last is embedding ASTs using GNNs [33,37], and here we take GCN as an example. As the number of GCN layers increases, each node can aggregate a more extensive range of information from its neighbors, thus focusing on a broader range of local features.…”

“…The treatment of ASTs in the previous studies mainly includes traversing them into sequences or using AST paths. Hu et al [23,24] used AST sequences obtained by a Structure-Based Traversal (SBT) method as input, which only extracts the AST structure globally without considering the relationships between nodes, losing the structural properties of ASTs [33]. In addition, LeClair et al [31] used GNN to embed the AST graph, but the embedding of ASTs using 2-Hop GNN was unable to extract the overall structure of ASTs.…”