Social Structure Construction from the Forums Using Interaction Coherence

Kashihara, Kazuaki; Shakarian, Jana; Baral, Chitta

doi:10.1007/978-3-030-63092-8_55

Cited by 5 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the importance of a post has very little utility when predicting interactions and thus social networks. More recently, Kashihara et al [12] proposed the Next Paragraph Prediction (NPP) method which extended BERT's Next Sentence Prediction to predict the response post from the previous post. This method allows for the Reconstruction of social networks using thread structure prediction.…”

Section: Thread Structure Predictionmentioning

confidence: 99%

“…A curated Reddit dataset [12] was used to train and evaluate our proposed model. The Reddit dataset is ideally suited for thread structure analysis given its treelike structure within different threads.…”

Section: Datasetsmentioning

confidence: 99%

“…Traditional methods used to define social networks on unstructured forums such as Creator-oriented Network and Last Reply-oriented Network [13] are based upon temporal interaction assumptions that do not consider the full context of the user interactions based on the content of the posts. Kashihara et al previously introduced a powerful deep learning method called Next Paragraph prediction (NPP) designed to define social networks using posts from the Reddit forums [12]. The NPP method outperformed traditional methods as well as BERT's Next Sentence Prediction (NSP) [3] when defining social networks from posts.…”

Section: Introductionmentioning

confidence: 99%

“…Building upon the NPP method, we propose the Next Paragraph Prediction with Instructional Prompting (NPP-IP) that leverages cutting-edge Promptbased learning to assist in social network construction from posts. Using the Reddit dataset [12] consisting of over 105 threads and 1, 648 posts, we train and evaluate the model against both traditional methods as well as the original NPP method. In addition, we test the model using real unstructured hacker forums data, where 20 threads are manually annotated by human experts to identify interactions based on posts.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Prompt-Based Learning for Thread Structure Prediction in Cybersecurity Forums

Kashihara¹,

Pal²,

Baral³

et al. 2023

Preprint

View full text Add to dashboard Cite

With recent trends indicating cyber crimes increasing in both frequency and cost, it is imperative to develop new methods that leverage data-rich hacker forums to assist in combating ever evolving cyber threats. Defining interactions within these forums is critical as it facilitates identifying highly skilled users, which can improve prediction of novel threats and future cyber attacks. We propose a method called Next Paragraph Prediction with Instructional Prompting (NPP-IP) to predict thread structures while grounded on the context around posts. This is the first time to apply an instructional prompting approach to the cybersecurity domain. We evaluate our NPP-IP with the Reddit dataset and Hacker Forums dataset that has posts and thread structures of real hacker forums' threads, and compare our method's performance with existing methods. The experimental evaluation shows that our proposed method can predict the thread structure significantly better than existing methods allowing for better social network prediction based on forum interactions.

show abstract

Section: Thread Structure Predictionmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Prompt-Based Learning for Thread Structure Prediction in Cybersecurity Forums

Kashihara¹,

Pal²,

Baral³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Our work is related to event or process discovery in process modeling tasks (Epure et al, 2015;Honkisz et al, 2018;Qian et al, 2020;Hanga et al, 2020), but our goal is not finding specific events or actions from procedural texts. In addition, the recent research proposed a method to create the forum structures from an unstructured forum based on the contents of each post using BERT's Next Sentence Prediction (Kashihara et al, 2020). However, we focus on building flow graphs for procedural texts using GNNs.…”

Section: Related Workmentioning

confidence: 99%

Constructing Flow Graphs from Procedural Cybersecurity Texts

Pal¹,

Kashihara²,

Banerjee³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Following procedural texts written in natural languages is challenging. We must read the whole text to identify the relevant information or identify the instruction-flow to complete a task, which is prone to failures. If such texts are structured, we can readily visualize instruction-flows, reason or infer a particular step, or even build automated systems to help novice agents achieve a goal. However, this structure recovery task is a challenge because of such texts' diverse nature. This paper proposes to identify relevant information from such texts and generate information flows between sentences. We built a large annotated procedural text dataset (CTFW) in the cybersecurity domain (3154 documents). This dataset contains valuable instructions regarding software vulnerability analysis experiences. We performed extensive experiments on CTFW with our LM-GNN model variants in multiple settings. To show the generalizability of both this task and our method, we also experimented with procedural texts from two other domains (Maintenance Manual and Cooking), which are substantially different from cybersecurity. Our experiments show that Graph Convolution Network with BERT sentence embeddings outperforms BERT in all three domains.

show abstract