PalmTree: Learning an Assembly Language Model for Instruction Embedding

Li, Xuezixiang; Qu, Yu; Yin, Heng

doi:10.1145/3460120.3484587

Cited by 78 publications

(65 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tasks in the benchmark range from intrinsic ones, aimed at evaluating how the pre-trained BinBert captures the semantic of instructions and sequences, to extrinsic downstream tasks, in which we fine-tune BinBert for problems on assembly sequences and binary functions. In all our experiments BinBert raises the performance bar outperforming the current state of the art (including the recent PalmTree [19]) and specific solutions created for the binary similarity problem.…”

Section: Our Proposal: Binbertmentioning

confidence: 99%

“…a memory address of an unknown memory layout) [22]. To ameliorate this problem, a lot of effort has been devoted to instructions preprocessing [42,15,30,14,19,22]. The standard of the field is to substitute all memory addresses and immediates above a certain threshold value with special symbols (e.g.…”

Section: Preprocessing Of Assembly Instructionsmentioning

confidence: 99%

“…IMM). Another design choice is whether to consider the entire assembly instructions as a token (used in [22]), to split the assembly instructions into several tokens by separating opcodes and operands (used in [14]) or to use a more fine-grained split strategy [19]. Interestingly, no one used automatic tokenization such as WordPiece [39] that are standard in NLP.…”

Section: Preprocessing Of Assembly Instructionsmentioning

confidence: 99%

“…A common technique is to transform assembly instructions into representational embeddings vectors, similarly to what has been done in the Natural Language Processing (NLP) field with the word embedding revolution [24]. Several works [22,42,9,19] proposed refined techniques to transform a single instruction into a vector of real numbers while capturing its semantic (e.g. all vectors of arithmetic instructions are clustered in the vector space).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

BinBert: Binary Code Understanding with a Fine-tunable and Execution-aware Transformer

Artuso¹,

Mormando²,

Luna³

et al. 2022

Preprint

View full text Add to dashboard Cite

A recent trend in binary code analysis promotes the use of neural solutions based on instruction embedding models. An instruction embedding model is a neural network that transforms sequences of assembly instructions into embedding vectors. If the embedding network is trained such that the translation from code to vectors partially preserves the semantic, the network effectively represents an assembly code model.In this paper we present BinBert, a novel assembly code model. BinBert is built on a transformer pretrained on a huge dataset of both assembly instruction sequences and symbolic execution information. BinBert can be applied to assembly instructions sequences and it is fine-tunable, i.e. it can be re-trained as part of a neural architecture on task-specific data. Through fine-tuning, BinBert learns how to apply the general knowledge acquired with pre-training to the specific task.We evaluated BinBert on a multi-task benchmark that we specifically designed to test the understanding of assembly code. The benchmark is composed of several tasks, some taken from the literature, and a few novel tasks that we designed, with a mix of intrinsic and downstream tasks.Our results show that BinBert outperforms state-of-the-art models for binary instruction embedding, raising the bar for binary code understanding.

show abstract

Section: Our Proposal: Binbertmentioning

confidence: 99%

Section: Preprocessing Of Assembly Instructionsmentioning

confidence: 99%

Section: Preprocessing Of Assembly Instructionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

BinBert: Binary Code Understanding with a Fine-tunable and Execution-aware Transformer

Artuso¹,

Mormando²,

Luna³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this paper, we propose a word embedding, and splicing mechanism based on Bi-LSTM (Li et al, 2021) and ConvLSTM, called Bi-direction Convolutional Word Embedding Classification Framework (BCDF). It can not only express the word vector in the text, but also embed the part of speech tagging information as a feature of sentence representation.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Text Emotion Analysis for Legal Anomie

She

2022

Front. Psychol.

View full text Add to dashboard Cite

Text emotion analysis is an effective way for analyzing the emotion of the subjects’ anomie behaviors. This paper proposes a text emotion analysis framework (called BCDF) based on word embedding and splicing. Bi-direction Convolutional Word Embedding Classification Framework (BCDF) can express the word vector in the text and embed the part of speech tagging information as a feature of sentence representation. In addition, an emotional parallel learning mechanism is proposed, which uses the temporal information of the parallel structure calculated by Bi-LSTM to update the storage information through the gating mechanism. The convolutional layer can better extract certain components of sentences (such as adjectives, adverbs, nouns, etc.), which play a more significant role in the expression of emotion. To take advantage of convolution, a Convolutional Long Short-Term Memory (ConvLSTM) network is designed to further improve the classification results. Experimental results show that compared with traditional LSTM model, the proposed text emotion analysis model has increased 3.3 and 10.9% F1 score on psychological and news text datasets, respectively. The proposed CBDM model based on Bi-LSTM and ConvLSTM has great value in practical applications of anomie behavior analysis.

show abstract