Multi-Label Classification of Historical Documents by Using Hierarchical Attention Networks

Abstract

The quantitative analysis of digitized historical documents has begun in earnest in recent years. Text classification is of particular importance for quantitative historical analysis because it helps to search literature efficiently and to determine the important subjects of a particular age. While numerous historians have joined together to classify large-scale historical documents, consistent classification among individual researchers has not been achieved. In this study, we present a classification method for large-scale historical data that uses a recently developed supervised learning algorithm called the Hierarchical Attention Network (HAN). By applying various classification methods to the Annals of the Joseon Dynasty (AJD), we show that HAN is more accurate than conventional techniques with word-frequency-based features. HAN provides the extent that a particular sentence or word contributes to the classification process through a quantitative value called 'attention'. We extract the representative keywords from various categories by using the attention mechanism and show the evolution of the keywords over the 472-year span of the AJD. Our results reveal that largely two groups of event categories are found in the AJD. In one group, the representative keywords of the categories were stable over long periods while the keywords in the other group varied rapidly, exhibiting repeatedly changing characteristics of the categories. Observing such macroscopic changes of representative words may provide insight into how a particular topic changes over a historical period.

Single vs. Multi-Label: The Issues, Challenges and Insights of Contemporary Classification Schemes

“…However, by stemming the vocabulary is significantly reduced and therefore the training time is also reduced. Kim [67] performed a multi-label classification on a Korean translated dataset and achieved accuracy up to 71%. Huang et al [68] used four datasets, Amazon Mobile Phone reviews, Amazon fine food reviews, and Yelp reviews 1 and 2.…”

A deep learning method DCWR with HANet for stock market prediction using news articles

“…Similarly, the attention structure is fed to allow the system to emphasize the LSTM or GRU outcomes accompanying with the arguments and lines that are most revealing of a specific class. We then established both models, i.e., LSTMs and GRUs through the hypermeter optimization process [40].…”

Using natural language processing methods to predict judicial outcomes

“…HANs are currently one of the most popular neural network algorithms being adopted in Computer Science. In 2020, the year of publication of this research, we have seen their applications in many different fields, such as Finance, to predict stock market values - Huang et al (2020); Linguistics, in the classification of historical documents - Kim et al (2020); Technology, in mobile app recommendations - Liang et al (2020); and Medicine, with detection models for Atrial Fibrillation -Mousavi, Afghah and Acharya (2020). We believe this flexibility shows the strength of the Attention model to handle different texts across many different vocabularies.…”