The Influence of Pre-processing on the Estimation of Readability of Web Documents

Abstract

This paper investigates the effect that text pre-processing approaches have on the estimation of the readability of web pages. Readability has been highlighted as an important aspect of web search result personalisation in previous work. The most widely used text readability measures rely on surface level characteristics of text, such as the length of words and sentences. We demonstrate that different tools for extracting text from web pages lead to very different estimations of readability. This has an important implication for search engines because search result personalisation strategies that consider users reading ability may fail if incorrect text readability estimations are computed.

Consumer Health Search on the Web: Study of Web Page Understandability and Its Integration in Ranking Algorithms (Preprint)

“…Using Justext or Boilerplate resulted in higher correlations with human understandability assessments, and the ForcePeriod heuristic was shown to be better than DoNotForcePeriod . These results confirm the speculations of Palotti et al [13]: they found these settings to produce lower variances in understandability estimations and thus hypothesised that they were better suited to the task.…”

“…Although no single setting outperformed the others in both collections, we found that the use of CLI and FRE with Justext provided the most stable results across the collections, with correlations as high as the best ones in both collections. These results confirmed the advice put forward by Palotti et al [13], i.e. in general, if using readability measures, then CLI is to be preferred, along with an appropriate HTML extraction pipeline, regardless of the heuristic for sentence ending.…”

“…We further study the influence of HTML processing methods on these estimations and their pitfalls, extending our previous work that has shown how this often ignored aspect greatly impacts effectiveness [13];…”

“…We did so by comparing the combination of a number of preprocessing pipelines, heuristics, and understandability estimation methods with human assessments of Web page understandability. Our experiments extended those by Palotti et al [13] and provided a much more thorough analysis, as they only evaluated surface level readability formulae and did not compare their results against human assessments.…”

“…These measures generally rely on surface-level characteristics of text, such as characters, syllables and word counts (missing citation). While these measures have been widely used in studies investigating the understandability of health content retrieved by search engines (e.g., [4,5,6,7,8,9,18,21]), our preliminary work found that these measures are heavily affected by the methods used to extract text from the HTML source [13]. We were able to identify specific settings of an HTML preprocessing pipeline that provided consistent estimates, but due to the lack of human assessments, we were not able to investigate how well each HTML preprocessing pipeline correlated with human assessments.…”