Predicting the quality of health web documents using their characteristics

Abstract

Purpose The quality of consumer-oriented health information on the web has been defined and evaluated in several studies. Usually it is based on evaluation criteria identified by the researchers and, so far, there is no agreed standard for the quality indicators to use. Based on such indicators, tools have been developed to evaluate the quality of web information. The HONcode is one of such tools. The purpose of this paper is to investigate the influence of web document features on their quality, using HONcode as ground truth, with the aim of finding whether it is possible to predict the quality of a document using its characteristics. Design/methodology/approach The present work uses a set of health documents and analyzes how their characteristics (e.g. web domain, last update, type, mention of places of treatment and prevention strategies) are associated with their quality. Based on these features, statistical models are built which predict whether health-related web documents have certification-level quality. Multivariate analysis is performed, using classification to estimate the probability of a document having quality given its characteristics. This approach tells us which predictors are important. Three types of full and reduced logistic regression models are built and evaluated. The first one includes every feature, without any exclusion, the second one disregards the Utilization Review Accreditation Commission variable, due to it being a quality indicator, and the third one excludes the variables related to the HONcode principles, which might also be indicators of quality. The reduced models were built with the aim to see whether they reach similar results with a smaller number of features. Findings The prediction models have high accuracy, even without including the characteristics of Health on the Net code principles in the models. The most informative prediction model considers characteristics that can be assessed automatically (e.g. split content, type, process of revision and place of treatment). It has an accuracy of 89 percent. Originality/value This paper proposes models that automatically predict whether a document has quality or not. Some of the used features (e.g. prevention, prognosis or treatment) have not yet been explicitly considered in this context. The findings of the present study may be used by search engines to promote high-quality documents. This will improve health information retrieval and may contribute to reduce the problems caused by inaccurate information.

Machine and cognitive intelligence for human health: systematic review

“…In terms of algorithm and method innovations, future efforts include: (1) using knowledge graphs to analyze medical information strongly relevant to expert knowledge to boost prediction performance [ 86 ], (2) adopting ontology-based methods to perform complex plan-oriented counseling and communication tasks [ 84 ], (3) utilizing CNNs and LSTM-CNN with diverse embedding and optimization technologies for epidemic outbreak analysis [ 44 ], (4) applying semi-automatic approaches to promote personalized healthcare information provided to facilitate users’ daily activities of living [ 76 ], (5) integrating additional security technologies like hashing to avoid malicious attackers [ 57 ], (6) using a combination of algorithms such as genetic algorithms and SVMs to facilitate accurate feature selection [ 55 ], and (7) adding more privacy-preserving statistics and machine learning algorithms to extensively promote flexibility in secure multicenters [ 57 ]. Additionally, there are scholars indicating the need to: (1) propose visual approaches to explore “the dyadic interaction between coaches and participants to better understand how to provide support and guidance to participants (p. 14) [ 61 ]”, and (2) analyze, mine, and extract Web page content by adopting machine learning algorithms and through quality information visualization within search engines [ 87 ].…”

“…In terms of model or system evaluation, future efforts need to focus on: (1) performing clinical trials to understand the impact of AI classifiers on skin cancer classification in real-life settings [ 72 ], (2) comparing with other predictive models via tenfold validation [ 98 ], (3) evaluating LSTM’s performance with Glove and Fasttext [ 44 ], (4) performing extrinsic evaluation focusing on the system’s ability “for high-risk findings in patient records and its impact on patient care and clinical decision-making (p. 318) [ 56 ]”, (5) validate summarization strategies with varied types of clinical texts (e.g., operative notes and radiology reports) in patient healthcare settings [ 56 ], (6) understanding how interaction patterns impact treatment benefits in Internet-based interventions [ 61 ], (7) exploring drug and side effect relationship extraction and adverse drug reaction extraction [ 62 ], (8) system assessment from different perspectives (e.g., group size and user profile size) [ 69 ], (9) studying the significance of adopting single variable versus multiple variables by search engines [ 87 ], and (10) conducting biological and clinical experiments to validate utility and effectives [ 54 ].…”

“…Additionally, other directions include: (1) practical use, for example, evaluating clinical decision support systems in real-world contexts to see how effective it supports clinical diagnosis and diminish diagnostic errors [ 31 ], adopting federated learning for actual medical data analysis through collaborations with multiple institutions [ 100 ], and designing Web-based tools for predicting the risk group of patients [ 67 ]; (2) automation, for example, automating the processes for improving data mining rules (e.g., rules acquired after implementing NLP technologies on Web data) [ 89 ], exploiting deep learning for automatic extraction of features from substantial log files [ 55 ], and automatic processing to infer health Web information quality to improve information retrieval [ 87 ]; (3) efficiency, for example, accelerating response time to enhance user experience [ 52 ], lengthening individual deployment duration to detect symptom occurrences with higher possibilities [ 102 ], and adopting graphics processing unit and field programmable gate array acceleration [ 57 ]; (4) data acquirement and quality improvement, for example, seizing physiological responses or symptoms for the exploration of the causal relationships between trigger exposure and asthma exacerbation [ 102 ], and constructing an inclusive database containing information on drug properties, dosage, and interactions related to depression medication [ 53 ]; (5) allowing interaction, for example, upgrading Web applications to allow users to input blood sample results along with the outcomes [ 101 ]; and (6) facilitating collaboration, for example, collaborating governmental healthcare sectors to construct text guidelines [ 27 ].…”

Googling for Abortion: Search Engine Mediation of Abortion Accessibility in the United States

“…Similar assessments of search result quality have been recently performed for searches related to mental health, suicide (Borge et al, 2021;Haim et al, 2017), and urolithiasis (kidney, bladder or urethra stones) (Chang et al, 2016). A general -related web pages has also been proposed (Oroszlányová et al, 2018), and researchers with access to search engine logs have focused on queries to understand user information needs (Abebe et al, 2019).…”

Search engines that help people make better health decisions

“…Another approach to appraise health information quality is by detecting the presence of multiple measurable indicators of quality [190,39,27,146,145]. Wang and Liu [190] In this thesis, we do not address problems related to quality appraisal, but we regard the creation of automated systems for appraising the quality of health webpage as an important direction for future work.…”