SECTOR: A Neural Model for Coherent Topic Segmentation and Classification

Abstract

When searching for information, a human reader first glances over a document, spots relevant sections and then focuses on a few sentences for resolving her intention. However, the high variance of document structure complicates to identify the salient topic of a given section at a glance. To tackle this challenge, we present SECTOR, a model to support machine reading systems by segmenting documents into coherent sections and assigning topic labels to each section. Our deep neural network architecture learns a latent topic embedding over the course of a document. This can be leveraged to classify local topics from plain text and segment a document at topic shifts. In addition, we contribute WikiSection, a publicly available dataset with 242k labeled sections in English and German from two distinct domains: diseases and cities. From our extensive evaluation of 20 architectures, we report a highest score of 71.6% F1 for the segmentation and classification of 30 topics from the English city domain, scored by our SECTOR LSTM model with bloom filter embeddings and bidirectional segmentation. This is a significant improvement of 29.5 points F1 compared to state-of-the-art CNN classifiers with baseline segmentation. 1 Our source code is available under the Apache License 2.0 at https

Learning Contextualized Document Representations for Healthcare Answer Retrieval

“…This approach places words with similar semantics nearby in vector space and allows queries with morphologic variations using subword representations. To find all possible aspects, we adopt prior work [4] and collect all section headings from the medical Wikipedia articles. These headings typically consist of 1-3 words and describe the main topic of a section.…”

“…We adopt the architecture of SECTOR [4] and use bidirectional LSTMs to read the document sentence-by-sentence. We use a final dense layer (matrix W he and bias b e ) to produce the local discourse vectors δ 1...T for every sentence in D.…”

A Large Test Collection for Entity Aspect Linking

“…Fetahu et al [9] extend Wikipedia articles on news events with up-to-date information from the web. Arnold et al [1,2] use sections on Wikipedia articles to (1) learn how to segment articles into different topics and (2) identify answer passages to biomedical questions.…”

“…We use a Wikipedia dump that is offered by the TREC Complex Answer Retrieval track, which exposes section and hyperlink information in a machine-readable format [8]. 1 If necessary, entity links for a given text passage can be readily created with an entity linking tool.…”

Aspect-Based Passage Retrieval with Contextualized Discourse Vectors

“…To complement the experimental evaluation in the original paper, we showcase three demonstrators to prove the usefulness of our approach in a real world scenario. The demos are available online using CORD-19 5 [18], WikiSection 6 [3], and Orphanet 7 [9] as their resource accordingly. Due to space constraints, we focus on the first system here.…”