Evaluating <scp>YouTube</scp> videos on sialendoscopy as an educational resource for patients

Campbell, Daniel; Xiao, Kevin; Mastrolonardo, Eric; Cognetti, David M.

doi:10.1002/lio2.991

Cited by 4 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study found that ChatGPT responses exceeded this grade level recommendation, averaging an FK score of 14.66, a collegiate reading level. Prior studies assessing ChatGPT’s grade level when responding to healthcare-related questions have found similar results [ 8 - 10 ]. Compared to other online sources, ChatGPT’s readability is on par.…”

Section: Discussionsupporting

confidence: 55%

Evaluating ChatGPT Responses on Atrial Fibrillation for Patient Education

Lee,

Campbell,

Rao

et al. 2024

Cureus

View full text Add to dashboard Cite

BackgroundChatGPT is a language model that has gained widespread popularity for its fine-tuned conversational abilities. However, a known drawback to the artificial intelligence (AI) chatbot is its tendency to confidently present users with inaccurate information. We evaluated the quality of ChatGPT responses to questions pertaining to atrial fibrillation for patient education. Our analysis included the accuracy and estimated grade level of answers and whether references were provided for the answers. MethodologyChatGPT was prompted four times and 16 frequently asked questions on atrial fibrillation from the American Heart Association were asked. Prompts included Form 1 (no prompt), Form 2 (patient-friendly prompt), Form 3 (physician-level prompt), and Form 4 (prompting for statistics/references). Responses were scored as incorrect, partially correct, or correct with references (perfect). Flesch-Kincaid grade-level unique words and response lengths were recorded for answers. Proportions of the responses at differing scores were compared using the chi-square analysis. The relationship between form and grade level was assessed using the analysis of variance. ResultsAcross all forms, scoring frequencies were one (1.6%) incorrect, five (7.8%) partially correct, 55 (85.9%) correct, and three (4.7%) perfect. Proportions of responses that were at least correct did not differ by form (p = 0.350), but perfect responses did (p = 0.001). Form 2 answers had a lower mean grade level (12.80 ± 3.38) than Forms 1 (14.23 ± 2.34), 3 (16.73 ± 2.65), and 4 (14.85 ± 2.76) (p < 0.05). Across all forms, references were provided in only three (4.7%) answers. Notably, when additionally prompted for sources or references, ChatGPT still only provided sources on three responses out of 16 (18.8%). ConclusionsChatGPT holds significant potential for enhancing patient education through accurate, adaptive responses. Its ability to alter response complexity based on user input, combined with high accuracy rates, supports its use as an informational resource in healthcare settings. Future advancements and continuous monitoring of AI capabilities will be crucial in maximizing the benefits while mitigating the risks associated with AI-driven patient education.

show abstract

Section: Discussionsupporting

confidence: 55%

Evaluating ChatGPT Responses on Atrial Fibrillation for Patient Education

Lee,

Campbell,

Rao

et al. 2024

Cureus

View full text Add to dashboard Cite

show abstract

“…Within digital medial there is an assumption that popularity of content correlates to quality and reliability of information. 17,22,23 Our study found no correlation between video quality and popularity. However, videos closer to the top of the YouTube search position had a positive correlation with our NI score.…”

Section: Discussionmentioning

confidence: 56%

“…Although we found audio-visual quality to have significant correlation, prior studies found a variety of results ranging from positive correlation to no correlation. [17][18][19] Video duration was also associated with more accurate and reliable YouTube videos. We found that longer videos included more content, and therefore were able to score higher on the information quality metrics that were implemented in this study.…”

Section: Discussionmentioning

confidence: 99%

“…Based on previous literature, qualitative score cut-offs were implemented for the modified DISCERN criteria: very poor (<9), poor (9-13), average (14)(15)(16)(17), good (18)(19)(20)(21), and excellent (22)(23)(24)(25). 11,12 Qualitative NI score cut-offs were similarly implemented: very poor (0-2.5), poor (2.6-4.0), average (4.1-6.0), good (6.1-8.0), excellent (8.1-10).…”

Section: Statistical Analysesmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating YouTube Videos as a High-Quality Educational Resource for Patients Contemplating Receiving Implantable Cardioverter Defibrillators

Lee,

Campbell,

Dayal

et al. 2024

Preprint

View full text Add to dashboard Cite

Introduction Patients increasingly use online sources to research treatment options and YouTube is the most used online video source. An estimated 40% of Americans watch YouTube videos regarding health and medicine; however, YouTube is unregulated and non-peer-reviewed. Methods We searched for YouTube videos using the term “Implantable Cardioverter Defibrillator.” Videos were categorized by uploader type and assessed using two methodologies: the modified DISCERN criteria (range: 0-25) and an electrophysiologist-reviewed Novel ICD (NI) score (range: 0-10). Qualitative score cut-offs were implemented stratifying the videos as: very poor, poor, average, good, and excellent. Secondary outcomes such as measures of popularity, duration, and audio-visual quality were collected and analyzed. Results Overall video quality was poor and highly variable across both scoring systems: mean modified DISCERN = 13.19±5.12; mean NI score = 4.09±2.56. Videos uploaded from academic institutions had a significantly higher modified DISCERN score than medical device companies, 13.97 (±4.59) vs. 9.039 (±5.16) (p=0.011). A significant positive correlation was found between video duration and both NI score (r=0.77, p<0.001) and modified DISCERN criteria (r=0.57, p<0.001). None of the metrics of video popularity correlated with video quality. Conclusion YouTube videos regarding ICDs available to patients were of a low and highly-variable quality. Factors such as uploader type, duration, and search position correlated with video quality. Although longer videos were more informative, videos that were optimal in length for attention (6 to 9 minutes) were variable in quality. More popular videos were not of higher quality, indicating that video preference by patients can be flawed.

show abstract

“…Indeed, two-thirds of TURP videos in our analysis were produced in 2021–2022. However, such online resources are generally not curated by medical experts, so whether viewers are learning correct medical techniques and principles is unclear ( 16 - 22 ). Here we performed what appears to be the first assessment of the educational quality of TURP videos on YouTube, for which we custom-designed a quality checklist.…”

Section: Discussionmentioning

confidence: 99%

The new checklist to evaluate educational value of YouTube surgical videos of transurethral resection of the prostate

Dong,

Peng,

Chen

et al. 2023

Transl Androl Urol

View full text Add to dashboard Cite

Background Transurethral resection of the prostate (TURP) is a widespread, effective way to treat benign prostatic hyperplasia (BPH). Many medical students and junior clinicians increasingly turn to easily accessible online resources to learn this technique, such as videos on YouTube. This study assessed the educational value of YouTube videos about TURP, which are popular among many young surgeons. Methods We searched YouTube as of August 2, 2022 for videos fulfilling the search terms “transurethral resection of the prostate”, “benign prostatic hyperplasia”, “BPH”, “TURP”, “benign prostatic enlargement”, “bladder outlet obstruction” and “lower urinary tract symptom”. We assessed the educational value of the identified videos using a custom-designed checklist. Results We identified 47 relevant videos, 20 of which were posted after July 1, 2020. The average number of views was 576,379±208,535 (range, 54–1,385,713). The average quality score of the videos was 7.38±2.53 (range, 4–12) on a 15-point scale, and 20 were judged to be of low educational quality. Quality scores correlated positively with the number of likes (R=0.596, P<0.01). Conclusions The educational value of most TURP videos on YouTube appears to be low, with most lacking detailed explanations of preoperative preparations and the surgical procedure. High-quality video resources about TURP need to be developed for medical students and junior surgeons. Standard quality criteria should also be developed and disseminated to ensure the production of accurate learning resources for junior clinicians.

show abstract

Evaluating YouTube videos on sialendoscopy as an educational resource for patients

Cited by 4 publications

References 33 publications

Evaluating ChatGPT Responses on Atrial Fibrillation for Patient Education

Evaluating ChatGPT Responses on Atrial Fibrillation for Patient Education

Evaluating YouTube Videos as a High-Quality Educational Resource for Patients Contemplating Receiving Implantable Cardioverter Defibrillators

The new checklist to evaluate educational value of YouTube surgical videos of transurethral resection of the prostate

Contact Info

Product

Resources

About