scite: A smart citation index that displays the context of citations and classifies their intent using deep learning

Nicholson, Joshua M.; Mordaunt, Milo; Lopez, Patrice; Uppala, Ashish; Rosati, Domenic; Rodrigues, Neves P.; Grabitz, Peter; Rife, Sean C.

doi:10.1162/qss_a_00146

Cited by 78 publications

(35 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This study uniquely compares the 2 platforms and highly suggests newer AI platforms will still need to be carefully scrutinized and validated when relied on for health information, abstract generation, or citation generation. Other platforms not evaluated in this report may be more carefully curated for academic purposes, such as Scite AI, 20 and have less of a hallucination rate. Additionally, this study is limited by the input of questions we used in each ophthalmic discipline.…”

Section: Limitationsmentioning

confidence: 99%

Evaluation and Comparison of Ophthalmic Scientific Abstracts and References by Current Artificial Intelligence Chatbots

et al. 2023

View full text Add to dashboard Cite

ImportanceLanguage-learning model–based artificial intelligence (AI) chatbots are growing in popularity and have significant implications for both patient education and academia. Drawbacks of using AI chatbots in generating scientific abstracts and reference lists, including inaccurate content coming from hallucinations (ie, AI-generated output that deviates from its training data), have not been fully explored.ObjectiveTo evaluate and compare the quality of ophthalmic scientific abstracts and references generated by earlier and updated versions of a popular AI chatbot.Design, Setting, and ParticipantsThis cross-sectional comparative study used 2 versions of an AI chatbot to generate scientific abstracts and 10 references for clinical research questions across 7 ophthalmology subspecialties. The abstracts were graded by 2 authors using modified DISCERN criteria and performance evaluation scores.Main Outcome and MeasuresScores for the chatbot-generated abstracts were compared using the t test. Abstracts were also evaluated by 2 AI output detectors. A hallucination rate for unverifiable references generated by the earlier and updated versions of the chatbot was calculated and compared.ResultsThe mean modified AI-DISCERN scores for the chatbot-generated abstracts were 35.9 and 38.1 (maximum of 50) for the earlier and updated versions, respectively (P = .30). Using the 2 AI output detectors, the mean fake scores (with a score of 100% meaning generated by AI) for the earlier and updated chatbot-generated abstracts were 65.4% and 10.8%, respectively (P = .01), for one detector and were 69.5% and 42.7% (P = .17) for the second detector. The mean hallucination rates for nonverifiable references generated by the earlier and updated versions were 33% and 29% (P = .74).Conclusions and RelevanceBoth versions of the chatbot generated average-quality abstracts. There was a high hallucination rate of generating fake references, and caution should be used when using these AI resources for health education or academic purposes.

show abstract

Section: Limitationsmentioning

confidence: 99%

Evaluation and Comparison of Ophthalmic Scientific Abstracts and References by Current Artificial Intelligence Chatbots

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In order to rectify this, Twitter and other digital platforms might take inspiration from scite, a recently developed tool aimed at tracking citations of articles, including retracted ones [44]. This tool has already been incorporated into citation management software like Zotero.…”

Section: Retractions As a Bellwether For Twitter Healthmentioning

confidence: 99%

Twitter Engagement with Retracted Articles: Who, When, and How?

Abhari¹,

Vincent²,

Dambanemuya³

et al. 2022

Preprint

View full text Add to dashboard Cite

Retracted research discussed on social media can spread misinformation, yet we lack an understanding of how retracted articles are mentioned by academic and non-academic users. This is especially relevant on Twitter due to the platform's prominent role in science communication. Here, we analyze the pre and post retraction differences in Twitter engagement metrics and content of mentions for over 3,800 retracted English-language articles alongside comparable non-retracted articles. We subset these findings according to the five user types detected by our supervised learning classifier: members of the public, scientists, bots, practitioners, and science communicators. We find that retracted articles receive greater overall engagement than non-retracted articles, especially among members of the public and bot users, the majority of engagement happening prior to retraction. Our results highlight non-scientists' involvement in retracted article discussions and suggest an opportunity for Twitter to include a retraction notice feature.

show abstract

“…The first results of content-based analysis in practice have already started to appear. A deep learning tool called scite was launched for classifying citation contexts (Nicholson et al, 2021).…”

Section: A Need To Go Beyond Counting: Content-based Citation Analysismentioning

confidence: 99%

Are papers published in predatory journals worthless? A geopolitical dimension revealed by content-based analysis of citations

Krawczyk¹,

Kulczycki²

2022

Preprint

View full text Add to dashboard Cite

This study uses content-based citation analysis to move beyond the simplified category of predatory (or questionable) journals. We present that when we analyze papers not only in terms of the number of their citations but also the content of these citations, we are able to show the much more complicated role of papers published in journals accused of being predatory. We analyzed the content of 9,995 citances from 6,706 papers indexed in the Web of Science Core Collection, which cites papers published in so-called questionable journals. The analysis revealed that the vast majority of such citances are neutral (97.3%), and negative citations of articles from questionable journals are almost completely nonexistent (0.8%). Moreover, the analysis revealed that the most frequently mentioned countries in the citances are India, Pakistan, and Iran, and mentions of Western countries are rare. This highlights geopolitical bias and shows the usefulness of looking at such journals as mislocated centers of scholarly communication. Apparently, the analyzed journals provide data needed in mainstream scholarly discussions, and the idea of predatory publishing hides geopolitical inequalities in global scholarly publishing. These findings also contribute to the further development of content-based citation analysis.

show abstract

scite: A smart citation index that displays the context of citations and classifies their intent using deep learning

Cited by 78 publications

References 41 publications

Evaluation and Comparison of Ophthalmic Scientific Abstracts and References by Current Artificial Intelligence Chatbots

Evaluation and Comparison of Ophthalmic Scientific Abstracts and References by Current Artificial Intelligence Chatbots

Twitter Engagement with Retracted Articles: Who, When, and How?

Are papers published in predatory journals worthless? A geopolitical dimension revealed by content-based analysis of citations

Contact Info

Product

Resources

About