Understanding large text corpora via sparse machine learning

Abstract

Sparse machine learning has recently emerged as powerful tool to obtain models of high-dimensional data with high degree of interpretability, at low computational cost. The approach has been successfully used in many areas, such as signal and image processing. This paper posits that these methods can be extremely useful in the analysis of large collections of text documents, without requiring user expertise in machine learning. Our approach relies on three main ingredients: (a) multi-document text summarization and (b) comparative summarization of two corpora, both using sparse regression or classification; (c) sparse principal components and sparse graphical models for unsupervised analysis and visualization of large text corpora. We validate our methods using a corpus of Aviation Safety Reporting System (ASRS) reports and demonstrate that the methods can reveal causal and contributing factors in runway incursions. Furthermore, we show that the methods automatically discover four main tasks that pilots perform during flight, which can aid in further understanding the causal and contributing factors to runway incursions and other drivers for aviation safety incidents. We also provide a comparative study involving other commonly used datasets, and report on the competitiveness of sparse machine learning compared to state-of-the-art methods such as Latent Dirichlet Allocation (LDA).

Instant‐Expertise in Failure Causation: Developing and Presenting a Network of Causes and Recommendations Extracted from Past Failures

“…While adding findings to the network is relatively easy, extracting and coding them requires more effort and time. Past research has been conducted on using machine learning to analyze accident reports [Abedin et al, 2010;Ghaoui et al, 2013;Robinson et al, 2015] so we may be able to use a similar method to teach a machine our coding scheme and easily add systems engineering failures to our solution aid.…”

Iterative Hard Thresholding for Keyword Extraction from Large Text Corpora

“…We observe that IHT, in [17] is shown to solve a very similar problem to the one that we look to solve. In fact, their formulations are identical to (2), however the presentation of the algorithm's convergence and performance guarantees in [17] are restricted to a small set of matrices: those which are full column rank and meet the 3k-restricted isometry property [21].…”

“…Blumensath and Davies [17] present the iterative hard thresholding algorithm (IHT) as an efficient algorithm for finding a local minimum of the least-squares objective in (2). IHT has the advantage of explicitly working with the cardinality-constrained feasible set, instead of the 1 convex relaxation, controlled by λ as a proxy for the cardinality.…”

“…[2] uses a formulation based on the Lasso to extract keywords using a bag of words model to identify topic descriptions. [1] uses an 1 -penalized logistic regression formulation and demonstrates the quality of the keywords selected by investigating the human interpretability of the model in the experimental results.…”

“…We can also quickly summarize a topic, by representing it with a small set of keywords -which can be used both to understand the topic, and to efficiently query other relevant documents for that topic. Keyword extraction can also be used for investigative purposes, for example to suggest causal factors that influence plane safety by analysing ASRS flight reports [2], or to investigate public perception and bias in media reporting [1].…”