Providing access to the high-level content of line graphs from online popular media

Abstract

This paper presents extensions to Interactive_SIGHT (Summarizing Information Graphics Textually), a system developed to provide sight-impaired individuals with access to information graphics present in multimodal documents from popular media. SIGHT is a Web-based tool that automatically recognizes the high-level knowledge of a graphic and generates natural language text, so screen readers are able to access it. Prior to this work, the SIGHT system was able to process and generate text only for simple bar charts. However, for graphics that are represented with lines and groups of bars, only the message recognition module has been developed. This work presents the steps that have been taken in order to construct a brief natural language summary that conveys the most important high-level content of single line graphs. It describes how the features of a line graph are identified and contextualized; how the content selection strategy was chosen and implemented; and how the propositions selected to convey the knowledge in the graphic are organized, allowing the discourse to be as clear and coherent as possible.

Generating Summaries of Line Graphs

“…In order to select the most important features of the line graph that should be conveyed in the summary, the system represents the intended message and the visual features identified by a human subject experiment (Greenbacker, Carberry, & McCoy, 2011) using a graph. A centrality-based algorithm, which is an adapted version of PageRank (Page, Brin, Motwani, & Winograd, 1999), is then implemented to select the most important information (represented as nodes in the graph). This implementation allows semantic relationships between propositions to be represented on the edges of the graph.…”

“…The core of the content selection framework is to detect present outstanding visual features in the graphic, along with its intended message, in order to select nodes. Details in the content selection phase are available in the work presented at (P. S. Moraes, Carberry, & McCoy, 2013).…”

Improve the Accessibility of Tactile Charts

“…They only cover the description of visual and structural properties of a chart such as labels, axis scale or number of bars. Some approaches focus on textual descriptions which include high-level content of the data visualization, too, in order to provide an idea of chart's intention such as Demir et al [5] and Moraes et al [12]. Tactile charts may be more effectively for analyzing data than verbal descriptions or data tables [15].…”

Adapting Graph Summaries to the Users’ Reading Levels

“…For such cases, the system must further decide whether to organize the description of the trends (1) by the trends themselves -e.g. either in left to right order -when no trend is considered more important than the others; or (2) by importance -when a trend has a greater set of features selected for the discourse or it composes a candidate intended message, which augments the intended message (Moraes et al, 2013). In the latter case, if a piece of the graphic (trend) has significantly more features selected, meaning that it possesses a higher number of visually outstanding features, it will be described first, then followed by the other trends.…”

“…The set of remarkable features is different for different graphics. Previous work of ours (Moraes, Carberry, & McCoy, 2013) presents methods that capture these most important features and allow the composition of customized summaries for each graph. Thus, given a graphic, our previous work has resulted in a system that can produce a set of propositions to include in its summary.…”