Are Papers Asking Questions Cited More Frequently in Computer Science?

Abstract

In this article, we test the hypothesis that computer science papers asking questions (i.e., those with a question mark at the end of their title) are cited more frequently than those that do not have this property. To this end, we analyze a data set of almost two million records on computer science papers indexed in the Web of Science database and focus our investigation on the mean number of citations per paper of its specific subsets. The main finding is that the average number of citations per paper of the so-called “asking papers” is greater by almost 20% than that of other papers, and that this difference is statistically significant.

Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper

“…Economics articles from multiple journals tend to have 1.6 more citations when they include a question mark (Gnewuch & Wohlrabe, 2017). Similarly, computer science journal articles and conference papers (1945–2014) received 16% more citations when including a question mark in their titles (Fiala et al, 2021) and another study supported the association between question marks in the titles and citations (regression coefficient: 0.414) in the field of Software Engineering (Graf‐Vlachy et al, 2022). This suggests that questioning titles either attract readers or tend to be associated with citable content, perhaps because the question is answered in the text.…”

“…Many additional studies were also identified from previous meta‐analyses or reviews (e.g., Shen et al, 2021; Tahamtan et al, 2016; Xie et al, 2019). Citation chaining in Scopus or Google Scholar, either through checking cited references or citations to relevant studies, was necessary to identify additional papers with keywords not matching the queries such as “Are papers asking questions cited more frequently in Computer Science?” (Fiala et al, 2021), “Easy to read, easy to cite?” (Dowling et al, 2018) or “The quest for citations: Drivers of article impact” (Stremersch et al, 2007).…”

Text and Team: What Article Metadata Characteristics Drive Citations in Software Engineering?

“…Specifically, authors frequently pose questions (that they may or may not answer in the title), and thus use question marks, in an article's title. Research on citations in computer science overall [18] and news media [43] has shown that the use of questions increases engagement with an article ("the clickbait phenomenon" [42, p. 1301]). We propose that the presence of such questions make an article more interesting and thus more likely to be read and considered by readers.…”

“…We obtained a sample of article abstracts and titles from Clarivate's Web of Science (WoS, formerly known as Web of Knowledge), a database that has been used in recent studies analyzing textual characteristics and/or citations of articles in computer science, information systems, and other disciplines [11,18,25,26,53,54,73]. We opted for WoS since it is superior to Google Scholar in various ways [23,69] and we did not have access to Scopus.…”

A citation analysis examining geographical specificity in article titles

“…However, a disciplinary effect exists. Papers with titles that ask questions are cited more in the computer science literature (Fiala et al 2021) but no citation effects were found for question-type titles in an ecology journal (Fox and Burns 2015).…”