Integrated visual analysis of patterns in time series and text data - Workflow and application to financial data analysis

Abstract

In this article, we describe a workflow and tool that allows a flexible formation of hypotheses about text features and their combinations, which are significantly connected in time to quantitative phenomena observed in stock data. To support such an analysis, we combine the analysis steps of frequent quantitative and text-oriented data using an existing a priori method. First, based on heuristics, we extract interesting intervals and patterns in large time series data. The visual analysis supports the analyst in exploring parameter combinations and their results. The identified time series patterns are then input for the second analysis step, in which all identified intervals of interest are analyzed for frequent patterns co-occurring with financial news. An a priori method supports the discovery of such sequential temporal patterns. Then, various text features such as the degree of sentence nesting, noun phrase complexity, and the vocabulary richness, are extracted from the news items to obtain meta-patterns. Meta-patterns are defined by a specific combination of text features which significantly differ from the text features of the remaining news data. Our approach combines a portfolio of visualization and analysis techniques, including time, cluster, and sequence visualization and analysis functionality. We provide a case study and an evaluation on financial data where we identify important future work. The workflow could be generalized to other application domains such as data analysis of smart grids, cyber physical systems, or the security of critical infrastructure, where the data consist of a combination of quantitative and textual time series data.

A Survey on Visual Analysis Approaches for Financial Data

“…Stock data containing time series of share prices of companies over a long time have been extensively used in Financial data analyses, including trend, pattern, performance, and predictive analysis. It is notable that the number of stocks used in analyses ranges from a few to hundreds [Sim03, DG04, STKF07, DBBS08, SBVLK09, BM12, WJS*16] to several thousand stocks (5328 stocks in a visualization [AKpK96]). The stock data is often combined with news media data to extract additional knowledge for providing contextual information [WP10, TA03, WJS*16].…”

“…Many visualization systems incorporating SOM tend to use the generated mapping data when the clusters are placed. For example, the approach shown in [Sim03, STKF07, STKF07, WJS*16] presents how SOM can be used for clustering with stock data and Financial statement data [CRVC13], while the approach in [SE11] demonstrates how the mapping information can be utilized for placing clusters generated from multivariate economic indicators. There is a complementary work that provides a collection of SOM‐based applications [DK98].…”

Social Media Visual Analytics

“…They propose an uncertainty model to retrieve semantic information, important keywords and users. Wanner et al [WJS*16] identify interesting financial time series intervals and corresponding news feature by extracting keywords in the news and social media. Their system supports users to analyze the relationships between financial patterns and text.…”

“…With topic extraction and event timeline visualization, they identified several interesting events, such as the event at the revelation of JP Morgan Chase multi‐billion dollar trading loss (the London Whale) [RWD14]. Wanner et al [WJS*16] extract keywords in the financial news, detect news features, identify interesting financial time series intervals with financial statistic data and analyze the relationship between news and financial market. In short, topic extraction and event timeline summary are commonly used techniques, to identify the correlation of the social media messages/mood with the financial markets.…”

Stock Market Visualization

“…As opposed to fully automated black boxes that only require data. 5 For prior works on stock market visualization and related literature, see, e.g.,[Chang et al, 2007],[Csallner et al, 2003],[Dao et al, 2008],[Deboeck, 1997a,b],[Dwyer & Eades, 2002],[Eklund et al, 2003],[Huang et al, 2009],[Ingle & Deshmukh, 2017],[Jungmeister & Turo, 1992],[Keim et al, 2006],[Korczak & Łuszczyk, 2011],[Lin et al, 2005],[Ma, 2009],[Marghescu, 2007],[Novikova & Kotenko, 2014],[NYT, 2011],[Parrish, 2000],[Rehan et al, 2013],[Roberts, 2004],[Schreck et al, 2007] ,[SEC, 2014],[Šimunić, 2003],,[Vande Moere & Lau, 2007],[Wang & Han, 2015],[Wanner et al, 2016],[Wattenberg, 1999],[Ziegler et al, 2010]. 6 Apart from aggressive taxi drivers, who are still rather innocuous compared with drivers in some other big cities.…”