An Unsupervised Approach to Genuine Health Information Retrieval Based on Scientific Evidence

Upadhyay, Rishabh; Pasi, Gabriella; Viviani, Marco

doi:10.1007/978-3-031-20891-1_10

Cited by 6 publications

(8 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Upadhyay et al ( 2022 ), we proposed a multidimensional retrieval model for CHS, i.e., a model to retrieve health-related documents by considering multiple relevance dimensions together. In this paper, we extend this model with an explainability component.…”

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

“…In the Upadhyay et al ( 2022 ) model, topical relevance is obtained by using the well-known and standard Okapi BM25 model (Brin and Page, 1998 ), which produces a topicality score denoted as BM25( q, d ) for a query q and a document d . Formally:…”

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

“…where its ( q, d ) refers to the information truthfulness score of the document d for a specific query q , cos is the cosine similarity score , and w 1 , w 2 , …, w k are distinct weights assigned to each similarity score, such that ∑ w i = 1 and w i ≥ w i +1 (1 ≤ i ≤ k −1). These weights allow assigning greater emphasis to the similarity scores according to the rank of the articles j j retrieved with respect to q , as illustrated in detail in Upadhyay et al ( 2022 ).…”

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

“…Both weights assume values in the [0, 1] interval. 3 Figure 1 illustrates the high-level operation of the multidimensional IR model under the CHS task proposed in Upadhyay et al ( 2022 ).…”

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

See 3 more Smart Citations

Explainable online health information truthfulness in Consumer Health Search

Upadhyay

Knoth

Pasi

et al. 2023

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

IntroductionPeople are today increasingly relying on health information they find online to make decisions that may impact both their physical and mental wellbeing. Therefore, there is a growing need for systems that can assess the truthfulness of such health information. Most of the current literature solutions use machine learning or knowledge-based approaches treating the problem as a binary classification task, discriminating between correct information and misinformation. Such solutions present several problems with regard to user decision making, among which: (i) the binary classification task provides users with just two predetermined possibilities with respect to the truthfulness of the information, which users should take for granted; indeed, (ii) the processes by which the results were obtained are often opaque and the results themselves have little or no interpretation.MethodsTo address these issues, we approach the problem as an ad hoc retrieval task rather than a classification task, with reference, in particular, to the Consumer Health Search task. To do this, a previously proposed Information Retrieval model, which considers information truthfulness as a dimension of relevance, is used to obtain a ranked list of both topically-relevant and truthful documents. The novelty of this work concerns the extension of such a model with a solution for the explainability of the results obtained, by relying on a knowledge base consisting of scientific evidence in the form of medical journal articles.Results and discussionWe evaluate the proposed solution both quantitatively, as a standard classification task, and qualitatively, through a user study to examine the “explained” ranked list of documents. The results obtained illustrate the solution's effectiveness and usefulness in making the retrieved results more interpretable by Consumer Health Searchers, both with respect to topical relevance and truthfulness.

show abstract

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

“…Both weights assume values in the [0, 1] interval. 3 Figure 1 illustrates the high-level operation of the multidimensional IR model under the CHS task proposed in Upadhyay et al ( 2022 ).…”

Section: Explaining Information Truthfulness In Consumer Health Searchmentioning

confidence: 99%

See 2 more Smart Citations

Explainable online health information truthfulness in Consumer Health Search

Upadhyay

Knoth

Pasi

et al. 2023

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, new credibility-centred search methods and assessment measures are crucial for tackling open issues in health-related information retrieval [17]. In the past few years, information retrieval shared tasks, such as the Text REtrieval Conference (TREC) and the Conference and Labs of the Evaluation Forum (CLEF), have started evaluating quality-based systems for health corpora [18,19]. The CLEF eHealth Lab Series proposed a benchmark to evaluate models according to the relevance, readability, and credibility of the retrieved information [20].…”

Section: Introductionmentioning

confidence: 99%

Online health search via multi-dimensional information quality assessment based on deep language models

Zhang

Naderi

Mishra

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: The presence of widespread misinformation in Web resources and the limited quality control provided by search engines can lead to serious implications for individuals seeking health advice. Objective: We aimed to investigate a multi-dimensional information quality assessment model based on deep learning to enhance the reliability of online healthcare information search results. Methods: In this retrospective study, we simulated online health information search scenarios with a topic set of 35 different health-related inquiries and a corpus containing one billion Web documents from the April 2019 snapshot of Common Crawl. Using state-of-the-art pre-trained language models, we inferred the usefulness, supportiveness, and credibility quality dimensions of the retrieved documents for a given search query. Results: The usefulness model provided the largest distinction between help and harm compatibility documents with a difference of 0.053. The supportiveness model achieved the best harm compatibility (0.024), while the combination of usefulness, supportiveness, and credibility models achieved the best help and harm compatibility on helpful topics. Conclusions: Our results suggest that integrating automatic ranking models created for specific information quality dimensions can increase the effectiveness of health-related information retrieval for decision-making.

show abstract

A systematic review of multidimensional relevance estimation in information retrieval

Peikos,

Pasi

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

In information retrieval, relevance is perceived as a multidimensional and dynamic concept influenced by user, task, and domain factors. Relying on this perspective, researchers have introduced multidimensional relevance models addressing diverse search tasks across numerous knowledge domains. Through our systematic review of 72 studies, we categorize research based on domain specificity and the distinct relevance aspects employed for estimating multidimensional relevance. Moreover, we highlight the approaches used to aggregate scores related to these factors and rank information items. Our insights underline the importance of concise definitions and unified methods for estimating relevance factors within and across domains. Finally, we identify benchmark collections for evaluations based on multiple relevance aspects while underscoring the necessity for new ones. Our findings suggest that large language models hold considerable promise for shaping future research in this field, mainly due to their relevance labeling abilities.This article is categorized under: Application Areas > Science and Technology Technologies > Computational Intelligence

show abstract

An Unsupervised Approach to Genuine Health Information Retrieval Based on Scientific Evidence

Cited by 6 publications

References 33 publications

Explainable online health information truthfulness in Consumer Health Search

Explainable online health information truthfulness in Consumer Health Search

Online health search via multi-dimensional information quality assessment based on deep language models

A systematic review of multidimensional relevance estimation in information retrieval

Contact Info

Product

Resources

About