In search of meaning: Lessons, resources and next steps for computational analysis of financial discourse

Abstract

We critically assess mainstream accounting and finance research applying methods from computational linguistics (CL) to study financial discourse. We also review common themes and innovations in the literature and assess the incremental contributions of studies applying CL methods over manual content analysis. Key conclusions emerging from our analysis are: (a) accounting and finance research is behind the curve in terms of CL methods generally and word sense disambiguation in particular; (b) implementation issues mean the proposed benefits of CL are often less pronounced than proponents suggest; (c) structural issues limit practical relevance; and (d) CL methods and high quality manual analysis represent complementary approaches to analyzing financial discourse. We describe four CL tools that have yet to gain traction in mainstream AF research but which we believe offer promising ways to enhance the study of meaning in financial discourse. The four tools are named entity recognition (NER), summarization, semantics and corpus linguistics. K E Y W O R D S 10-K, annual reports, computational linguistics, conference calls, corpus linguistics, earnings announcements, machine learning, NLP, semantics 1Information is the lifeblood of financial markets and the amount of data available to decision-makers is increasing exponentially. Bank of England (2015) estimates that 90% of global information has been created during the last decade, (MD&A), whereas practitioners, standard setters and regulators are often interested in more granular issues such as the format and content of specific disclosures, placement of content within the overall reporting package, limits on the use of jargon concerning particular topics, etc. Second, it is not immediately obvious how commonly employed empirical proxies for discourse quality such as readability (Fog index), tone (word-frequency counts) and text re-use (cosine similarity) map into the practical properties of effective communication identified by financial market regulators.With these caveats in mind, we proceed to review common themes and innovations in the literature and assess the incremental contributions of work applying CL methods over manual content analysis. The median AF study examines 10-K filings using basic content analysis methods such as readability algorithms and keyword counts. The degree of clustering is consistent with the initial phase of the research lifecycle, with agendas shaped as much by ease of data access and implementation as by research priorities. Nevertheless, closer inspection reveals how relatively basic word-level methods have been used to provide richer insights into the properties and effects of financial discourse.Refinements to standard readability metrics, development of domain-specific wordlists, and the use of weighting schemes and text filtering to improve word-sense disambiguation represent welcome advances on naïve unigram word counts. We also acknowledge a move towards the use of more NLP technology in the form of machine learning and topic...

The “Walk” towards the UN Sustainable Development Goals: Does Mandated “Talk” through NonFinancial Disclosure Affect Companies’ Financial Performance?

“…As can be seen, this is simply a yes/no checklist of items that may or may not be disclosed. This manual collection of data is also consistent with the recent study of El Haj et al [110], which makes it clear that automated computer linguistics methods have so far provided little advantage over manual methods in terms of the generated insights.…”

Retrieving, classifying and analysing narrative commentary in unstructured (glossy) annual reports published as PDF files

“…Annual reports provide important information to support decision-making (CFA Society U.K. 2016; EY 2015: 6). 1 Extant large sample automated analysis of annual report commentaries focuses almost entirely on Form 10-K filings for U.S. registrants accessed through the Securities and Exchange Commission's (SEC) EDGAR system (El-Haj et al 2019). Several features make 10-Ks amenable to automated large-sample research including batch retrieval provisions, plain text formatting, and a standardised reporting template.…”

Using long short‐term memory neural networks to analyze SEC 13D filings: A recipe for human and machine interaction

“…Instead of using the more common bag‐of‐words approach, we study both the vocabulary and the structure of the sentences included in the filing. Recent research by El‐Haj et al () and others suggests that this is a more effective way to extract meaning from text.…”

“…El‐Haj, Rayson, Walker, Young, and Simaki () reviewed the research papers in accounting and finance that apply computational linguistics methods to the study of financial discourse. They provided a survey of common themes and innovations in the literature and evaluated the contributions of computational linguistic methods over manual textual analysis.…”