Analysing and evaluating the task of automatic tweet generation: Knowledge to business

Abstract

In this paper a study concerning the evaluation and analysis of natural language tweets is presented. Based on our experience in text summarisation, we carry out a deep analysis on user's perception through the evaluation of tweets manual and automatically generated from news. Specifically, we consider two key issues of a tweet: its informativeness and its interestingness. Therefore, we analyse: 1) do users equally perceive manual and automatic tweets?; 2) what linguistic features a good tweet may have to be interesting, as well as informative? The main challenge of this proposal is the analysis of tweets to help companies in their positioning and reputation on the Web. Our results show that: 1) automatically informative and interesting natural language tweets can be generated as a result of summarisation approaches; and 2) we can characterise good and bad tweets based on specific linguistic features not present in other types of tweets.Keywords: Natural Language Processing, Text Summarisation, Natural Language Tweet Generation, User Study, Linguistic Analysis, Descriptive Statistics Introduction, Context and MotivationIn the current digital knowledge society, the overload of information has become a problem to companies, which cannot cope with all the available information. As a consequence, companies may not be exploiting the Web, and taking advantage of it accordingly, thus affecting key aspects, such as their visibility, reputation, marketing campaigns, customer's feedback, etc. With the Preprint submitted to Computers in IndustryOctober 11, 2015 This is a previous version of the article published in Computers in Industry. 2016Industry. , 78: 3-15. doi:10.1016Industry. /j.compind.2015 birth of the Web 2.0, there has been a shift in the way the information is produced and consumed by users and companies. The Web 2.0 has established a wide range of on-line mechanisms and platforms through which companies can obtain direct feedback from users. These mechanisms (e.g., reviews, social net- With more than 241 million active users per month 1 , 184 million of which uses Twitter through their mobile device, and more than 500 million tweets daily 2 , Twitter 3 has become an excellent social media for on-line real-time news attention 4 . The length restriction imposed on tweets (140 characters) force messages to be concise, though it is also possible to link out to external information to enrich the tweet. Moreover, hashtags (e.g. #UA Universidad) allow to categorise information, to identify the trending topics, and more importantly to enable a rapid on-line information flow. According to [3]

Text analytics in industry: Challenges, desiderata and trends

“…the sentence splitter of the NLTK toolkit [19] is employed in the text clustering and sentiment analysis application of [12]. The application for analyzing aviation safety reports presented in [16] relies on the TreeTagger toolkit [32] for PoStagging and lemmatization, while the Freeling toolkit [25] was used in [13] to process tweets. The application for processing health messages on social media in [18] made extensive use of the annotation pipeline from the GATE toolkit [44].…”

“…The evaluation procedure of [13] deserves particular attention. In this study, the authors evaluate the application's performance by measuring how users perceive its end results.…”

“…Our review of the current state-of-the-art will be based on the scientific articles published in this special issue of the Computers in Industry journal 2 on text analytics in industry, namely [12][13][14][15][16][17][18].…”

Cross-domain informativeness classification for disaster situations

“…The concept of informativeness is diverse in its use and discussed in various areas, including e.g. informativeness of web documents [Horn et al, 2013], term informativeness [Wong and Kit, 2011] [Wu and Giles, 2013] or informativeness of Social Media messages in areas like news [Lloret and Palomar, 2016] as well as the crisis domain itself [Olteanu et al, 2015]. Yet, informativeness is a subjective concept, which heavily depends on the receiver of the information [Olteanu et al, 2015].…”

“…Yet, informativeness is a subjective concept, which heavily depends on the receiver of the information [Olteanu et al, 2015]. Since a variety of informativeness definitions exist [Derczynski et al, 2018] [Horn et al, 2013] [Lloret and Palomar, 2016] [Longhini et al, 2017], the current work follows the informativeness definition of Olteanu et al…”