Event Representation Learning Enhanced with External Commonsense Knowledge

Abstract

Prior work has proposed effective methods to learn event representations that can capture syntactic and semantic information over text corpus, demonstrating their effectiveness for downstream tasks such as script event prediction. On the other hand, events extracted from raw texts lacks of commonsense knowledge, such as the intents and emotions of the event participants, which are useful for distinguishing event pairs when there are only subtle differences in their surface realizations. To address this issue, this paper proposes to leverage external commonsense knowledge about the intent and sentiment of the event. Experiments on three event-related tasks, i.e., event similarity, script event prediction and stock market prediction, show that our model obtains much better event embeddings for the tasks, achieving 78% improvements on hard similarity task, yielding more precise inferences on subsequent events under given contexts, and better accuracies in predicting the volatilities of the stock market 1 .

Event-Driven Learning of Systematic Behaviours in Stock Markets

“…We propose two updates of the original HANs used in (Hu et al, 2018) and (Yang et al, 2016): inserting an event-layer between words and news levels, and representing events by pretrained contextualized language models such as BERT (Devlin et al, 2019) instead of bidirectional gated recurrent unit (bi-GRU) (Cho et al, 2014) plus attention networks. The encoding of financial information in the units of events is inspired by Ding et al (2014Ding et al ( , 2019. The difference is that, we replace the neural tensor network (Ding et al, 2014) by BERT.…”

“…(2) too long news prevents the generalization ability of being embedded for financial information similarity computing. Generally, our proposed network can be seen as a combination of events (Ding et al, 2014(Ding et al, , 2019 represented by deep pretrained contextualized language models (Devlin et al, 2019) inside a hierarchical attention mechanisms (Hu et al, 2018;Yang et al, 2016). In Figure 2, we assume that there are N words in one financial event.…”

“…We select Standard&Poor (S&P) 500, Dow Jones and Nasdaq indices 5 to disclose how far financial news information can bring impacts to the systematic behaviours in stock markets. Following researches on stock movement prediction (Hu et al, 2018;Ding et al, 2019), we define it as a multicategory classification problem.…”

“…It is widely reported that financial news, especially financial events expressed in the news, influence the movements of stock markets (Ding et al, 2014(Ding et al, , 2015(Ding et al, , 2016Hu et al, 2018;Ding et al, 2019;Huang et al, 2019;Glasserman et al, 2019). For example, one company's releasing of new products brings novel time-sensitive factors to the stock movement of that company and further to the whole market; changing of interest rates from the central bank of the country bring changes of currency fluidity invested in the stock market.…”

The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation

“…Just to name a few, Trinh and Le (2018), He et al (2019) and Klein and Nabi (2019) use language models trained on huge text corpora to do inference on the WSC dataset. Ding et al (2019) use commonsense knowledge in Atomic (Sap et al, 2019a) and Event2mind (Rashkin et al, 2018) on downstream tasks such as script event prediction. Bi et al (2019) exploit external commonsense knowledge from ConceptNet (Speer et al, 2016)) in machine reading comprehension.…”