When classification accuracy is not enough: Explaining news credibility assessment

Przybyła, Piotr; Soto, Axel J.

doi:10.1016/j.ipm.2021.102653

Cited by 24 publications

(14 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In another research, authors have used link-based, commercial, page rank, presentation, and word features to check the reliability of web pages in the medical domain [ 23 ]. Authors in [ 8 , 17 ] found user-based features, sentimental features, syntactic, part-of-speech, and propagation features more suitable for finding the credibility of contents. In [ 2 ], authors introduced novel linguistic features with persuasive, comparative, uncertain, and emotional words.…”

Section: Related Workmentioning

confidence: 99%

“…Fact-checking aims to assess the truthiness of information under investigation to identify whether the information is factual. The knowledge-based approach requires previously fact-checked data to determine the credibility of contents [ 8 ]. Fact-checking involves examining the facts to verify the legitimacy of the information [ 9 , 10 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting and classifying online health misinformation with ‘Content Similarity Measure (CSM)’ algorithm: an automated fact-checking-based approach

Barve¹,

Saini

2023

J Supercomput

View full text Add to dashboard Cite

Information dissemination occurs through the 'word of media' in the digital world. Fraudulent and deceitful content, such as misinformation, has detrimental effects on people. An implicit fact-based automated fact-checking technique comprising information retrieval, natural language processing, and machine learning techniques assist in assessing the credibility of content and detecting misinformation. Previous studies focused on linguistic and textual features and similarity measures-based approaches. However, these studies need to gain knowledge of facts, and similarity measures are less accurate when dealing with sparse or zero data. To fill these gaps, we propose a 'Content Similarity Measure (CSM)' algorithm that can perform automated fact-checking of URLs in the healthcare domain. Authors have introduced a novel set of content similarity, domain-specific, and sentiment polarity score features to achieve journalistic fact-checking. An extensive analysis of the proposed algorithm compared with standard similarity measures and machine learning classifiers showed that the ‘content similarity score’ feature outperformed other features with an accuracy of 88.26%. In the algorithmic approach, CSM showed improved accuracy of 91.06% compared to the Jaccard similarity measure with 74.26% accuracy. Another observation is that the algorithmic approach outperformed the feature-based method. To check the robustness of the algorithms, authors have tested the model on three state-of-the-art datasets, viz. CoAID, FakeHealth, and ReCOVery. With the algorithmic approach, CSM showed the highest accuracy of 87.30%, 89.30%, 85.26%, and 88.83% on CoAID, ReCOVery, FakeHealth (Story), and FakeHealth (Release) datasets, respectively. With a feature-based approach, the proposed CSM showed the highest accuracy of 85.93%, 87.97%, 83.92%, and 86.80%, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Detecting and classifying online health misinformation with ‘Content Similarity Measure (CSM)’ algorithm: an automated fact-checking-based approach

Barve¹,

Saini

2023

J Supercomput

View full text Add to dashboard Cite

show abstract

“…It also has built-in functionality for data input and a visualization API [106]. For example in [95], a browser extension for online news credibility assessment tools 10 was developed using a collection of interactive visualizations that explain the reasoning behind the automated credibility evaluation to the user. As another example, in [11], a neural network was developed and exported to the Arduino Nano Module for geometric object identification by their acoustic signatures, which attempts to replicate the way dolphins and bats perceive by using sonic waves.…”

Section: Robust Ecosystemmentioning

confidence: 99%

Front-end deep learning web apps development and deployment: a review

Goh

Abas

2022

Appl Intell

View full text Add to dashboard Cite

Machine learning and deep learning models are commonly developed using programming languages such as Python, C++, or R and deployed as web apps delivered from a back-end server or as mobile apps installed from an app store. However, recently front-end technologies and JavaScript libraries, such as TensorFlow.js, have been introduced to make machine learning more accessible to researchers and end-users. Using JavaScript, TensorFlow.js can define, train, and run new or existing, pre-trained machine learning models entirely in the browser from the client-side, which improves the user experience through interaction while preserving privacy. Deep learning models deployed on front-end browsers must be small, have fast inference, and ideally be interactive in real-time. Therefore, the emphasis on development and deployment is different. This paper aims to review the development and deployment of these deep-learning web apps to raise awareness of the recent advancements and encourage more researchers to take advantage of this technology for their own work. First, the rationale behind the deployment stack (front-end, JavaScript, and TensorFlow.js) is discussed. Then, the development approach for obtaining deep learning models that are optimized and suitable for front-end deployment is then described. The article also provides current web applications divided into seven categories to show deep learning potential on the front end. These include web apps for deep learning playground, pose detection and gesture tracking, music and art creation, expression detection and facial recognition, video segmentation, image and signal analysis, healthcare diagnosis, recognition, and identification.

show abstract

“…On the other hand, we face a more limited object of study, enabling us to build more expressive, more fine-grained representations of knowledge, for instance, in the form of ontologies [16,23] and datasets [59,53]. Besides, the reduced model complexity allows for easier automation, which is crucial in times of an ever-increasing distribution speed for newly created content [6]: If users can spread their texts in real time, they also need tools for maintaining credibility standards in real time, which is hard to achieve without a reasonable amount of computerization [47].…”

Section: Covid-19 As a Domain For Credibility Assessmentmentioning

confidence: 99%

User Experience Design for Automatic Credibility Assessment of News Content About COVID-19

Schulz¹,

Rauenbusch²,

Jan³

et al. 2022

Preprint

View full text Add to dashboard Cite

The increasingly rapid spread of information about COVID-19 on the web calls for automatic measures of credibility assessment [18]. If large parts of the population are expected to act responsibly during a pandemic, they need information that can be trusted [20].In that context, we model the credibility of texts using 25 linguistic phenomena, such as spelling, sentiment and lexical diversity. We integrate these measures in a graphical interface and present two empirical studies to evaluate its usability for credibility assessment on COVID-19 news. Raw data for the studies, including all questions and responses, has been made available to the public using an open license: https://github.com/ konstantinschulz/credible-covid-ux. The user interface prominently features three sub-scores and an aggregation for a quick overview. Besides, metadata about the concept, authorship and infrastructure of the underlying algorithm is provided explicitly. Our working definition of credibility is operationalized through the terms of trustworthiness, understandability, transparency, and relevance. Each of them builds on well-established scientific notions [41,65,68] and is explained orally or through Likert scales. In a moderated qualitative interview with six participants, we introduce information transparency for news about COVID-19 as the general goal of a prototypical platform, accessible through an interface in the form of a wireframe [43]. The participants' answers are transcribed in excerpts. Then, we triangulate inductive and deductive coding methods [19] to analyze their content. As a result, we identify rating scale, sub-criteria and algorithm authorship as important predictors of the usability. In a subsequent quantitative online survey, we present a questionnaire with wireframes to 50 crowdworkers. The question formats include Likert scales, multiple choice and open-ended types. This way, we aim to strike a balance between the known strengths and weaknesses of open vs. closed questions [11]. The answers reveal a conflict between transparency and conciseness in the interface design: Users tend to ask for more information, but do not necessarily make explicit use of it when given. This discrepancy is influenced by capacity constraints of the human working memory [38]. Moreover, a perceived hierarchy of metadata

show abstract

When classification accuracy is not enough: Explaining news credibility assessment

Cited by 24 publications

References 36 publications

Detecting and classifying online health misinformation with ‘Content Similarity Measure (CSM)’ algorithm: an automated fact-checking-based approach

Detecting and classifying online health misinformation with ‘Content Similarity Measure (CSM)’ algorithm: an automated fact-checking-based approach

Front-end deep learning web apps development and deployment: a review

User Experience Design for Automatic Credibility Assessment of News Content About COVID-19

Contact Info

Product

Resources

About