Online health search via multi-dimensional information quality assessment based on deep language models

Zhang, Boya; Naderi, Nona; Mishra, Rahul; Teodoro, Douglas

doi:10.1101/2023.04.11.22281038

Cited by 3 publications

(1 citation statement)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Questionnaires were then uploaded to the tool and were indexed using ElasticSearch. During the search phase, the tool automatically computes the similarity between the input query and the indexed documents using the TF-IDF framework, 26 28 more specifically the BM25 model, which also considers document length and term saturation. Then, the tool provides as results for a query a ranked list of questionnaires matching the search template.…”

Section: Methodsmentioning

confidence: 99%

Did high frequency phone surveys during the COVID-19 pandemic include disability questions? An assessment of COVID-19 surveys from March 2020 to December 2022

Nascimento Dial,

Vicente,

Mitra

et al. 2024

BMJ Open

Self Cite

View full text Add to dashboard Cite

ObjectivesIn the midst of the pandemic, face-to-face data collection for national censuses and surveys was suspended due to limitations on mobility and social distancing, limiting the collection of already scarce disability data. Responses to these constraints were met with a surge of high-frequency phone surveys (HFPSs) that aimed to provide timely data for understanding the socioeconomic impacts of and responses to the pandemic. This paper provides an assessment of HFPS datasets and their inclusion of disability questions to evaluate the visibility of persons with disabilities during the COVID-19 pandemic.DesignWe collected HFPS questionnaires conducted globally from the onset of the pandemic emergency in March 2020 until December 2022 from various online survey repositories. Each HFPS questionnaire was searched using a set of keywords for inclusion of different types of disability questions. Results were recorded in an Excel review log, which was manually reviewed by two researchers.MethodsThe review of HFPS datasets involved two stages: (1) a main review of 294 HFPS dataset-waves and (2) a semiautomated review of the same dataset-waves using a search engine-powered questionnaire review tool developed by our team. The results from the main review were compared with those of a sensitivity analysis using and testing the tool as an alternative to manual search.ResultsRoughly half of HFPS datasets reviewed and 60% of the countries included in this study had some type of question on disability. While disability questions were not widely absent from HFPS datasets, only 3% of HFPS datasets included functional difficulty questions that meet international standards. The search engine-powered questionnaire review tool proved to be able to streamline the search process for future research on inclusive data.ConclusionsThe dearth of functional difficulty questions and the Washington-Group Short Set in particular in HFPS has contributed to the relative invisibility of persons with disabilities during the pandemic emergency, the lingering effects of which could impede policy-making, monitoring and advocacy on behalf of persons with disabilities.

show abstract

Section: Methodsmentioning

confidence: 99%

Did high frequency phone surveys during the COVID-19 pandemic include disability questions? An assessment of COVID-19 surveys from March 2020 to December 2022

Nascimento Dial,

Vicente,

Mitra

et al. 2024

BMJ Open

Self Cite

View full text Add to dashboard Cite

show abstract

Beyond Topicality: Including Multidimensional Relevance in Cross-encoder Re-ranking

Upadhyay,

Askari,

Pasi

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Online Health Search Via Multidimensional Information Quality Assessment Based on Deep Language Models: Algorithm Development and Validation

Zhang,

Naderi,

Mishra

et al. 2024

JMIR AI

View full text Add to dashboard Cite

Background Widespread misinformation in web resources can lead to serious implications for individuals seeking health advice. Despite that, information retrieval models are often focused only on the query-document relevance dimension to rank results. Objective We investigate a multidimensional information quality retrieval model based on deep learning to enhance the effectiveness of online health care information search results. Methods In this study, we simulated online health information search scenarios with a topic set of 32 different health-related inquiries and a corpus containing 1 billion web documents from the April 2019 snapshot of Common Crawl. Using state-of-the-art pretrained language models, we assessed the quality of the retrieved documents according to their usefulness, supportiveness, and credibility dimensions for a given search query on 6030 human-annotated, query-document pairs. We evaluated this approach using transfer learning and more specific domain adaptation techniques. Results In the transfer learning setting, the usefulness model provided the largest distinction between help- and harm-compatible documents, with a difference of +5.6%, leading to a majority of helpful documents in the top 10 retrieved. The supportiveness model achieved the best harm compatibility (+2.4%), while the combination of usefulness, supportiveness, and credibility models achieved the largest distinction between help- and harm-compatibility on helpful topics (+16.9%). In the domain adaptation setting, the linear combination of different models showed robust performance, with help-harm compatibility above +4.4% for all dimensions and going as high as +6.8%. Conclusions These results suggest that integrating automatic ranking models created for specific information quality dimensions can increase the effectiveness of health-related information retrieval. Thus, our approach could be used to enhance searches made by individuals seeking online health information.

show abstract

Online health search via multi-dimensional information quality assessment based on deep language models

Cited by 3 publications

References 67 publications

Did high frequency phone surveys during the COVID-19 pandemic include disability questions? An assessment of COVID-19 surveys from March 2020 to December 2022

Did high frequency phone surveys during the COVID-19 pandemic include disability questions? An assessment of COVID-19 surveys from March 2020 to December 2022

Beyond Topicality: Including Multidimensional Relevance in Cross-encoder Re-ranking

Online Health Search Via Multidimensional Information Quality Assessment Based on Deep Language Models: Algorithm Development and Validation

Contact Info

Product

Resources

About