ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis

Abstract

Analyzing the readability of articles has been an important sociolinguistic task. Addressing this task is necessary to the automatic recommendation of appropriate articles to readers with different comprehension abilities, and it further benefits education systems, web information systems, and digital libraries. Current methods for assessing readability employ empirical measures or statistical learning techniques that are limited by their ability to characterize complex patterns such as article structures and semantic meanings of sentences. In this paper, we propose a new and comprehensive framework which uses a hierarchical self-attention model to analyze document readability. In this model, measurements of sentence-level difficulty are captured along with the semantic meanings of each sentence. Additionally, the sentence-level features are incorporated to characterize the overall readability of an article with consideration of article structures. We evaluate our proposed approach on three widely-used benchmark datasets against several strong baseline approaches. Experimental results show that our proposed method achieves the state-of-the-art performance on estimating the readability for various web articles and literature.C. Meng-This work was done during the summer internships of CM and MC at Google, Mountain View. We thank the anonymous reviewers for their insightful comments.

Assessment of the Clarity of Bank of Russia Monetary Policy Communication by Neural Network Approach

“…Table 3 shows the results of the models' operation on the test dataset (1,500 texts, about 150 texts per complexity level). The most successful (F1-score = 94.8%) is the Transformer neural network model, which is consistent with the results obtained by Meng (2020) for the English language.…”

“…The second approach is successfully applied, for example, by Meng et al (2020). The authors propose the pre-processing of a text using the ReadNet system of linguistic characteristics, which allows the neural network to 'decode' the text and handle the classification task a little faster.…”

Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features

“…But all hybrid models beat previous SOTA results by a large margin. Notably, we achieve the near-perfect accuracy of 99% on OneStopEnglish, a massive 20.3% increase from the previous SOTA (Martinc et al, 2021) by HAN (Meng et al, 2020).…”

“…Regarding the model architecture, we examined appending the handcrafted features to transformer embeddings without the use of a secondary predic- (Meng et al, 2020) hints that such a model is not robust to small datasets. ReadNet reports 52.8% accuracy on Cambridge, worse than any of our tested models (table 2, 3, 5).…”

Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)

“…Later machine learning methods for readability assessment may rely on feature extraction (e.g. Meng et al, 2020;Deutsch et al, 2020; also see for an analysis of different approaches).…”