Created in 1991, the Ig Nobel Prize is a satire of the official Nobel Prize, awarded in ten categories each year to “honor achievements that first make people laugh, and then make them think.” While not only awarded for scientific accomplishments, the majority of Ig Nobel Prize-winners receive the prize for published scientific articles.

Though most Ig Nobel research remains relatively obscure, some award-winning work has been highly cited and become quite famous — for example, Justin Kruger and David Dunning were awarded an Ig Nobel Prize in Psychology for their finding that incompetent people often overestimate their abilities, a finding later named the Dunning-Kruger Effect.

From the ridiculous to the mundane, the disgusting to the thought-provoking, Ig Nobel Prizes are awarded across many disciplines. For example, the Economics Prize was awarded to Kopczuk and Slemrod (2003) for finding that people attempt to live longer if it can get them a lower estate tax upon their death; and the Anatomy Prize to James Heathcote for his work “Why Do Old Men Have Big Ears?”. The set of prize-winning papers is a collection of some of the most outlandish yet interesting research that has been done over the past few decades.

For such a curious set of papers, we at scite were curious to see: other than receiving the Ig Nobel Prize, how have such scientific papers been discussed in the literature? Are they highly cited? Have their findings been supported by others, or disputed? We set out to answer some of these questions using scite’s Custom Dashboards.

An overview of the data

After collecting the DOIs of all Ig Nobel Prize-winning articles from Wikipedia, we created a Custom Dashboard to see how all Ig Nobel articles have been supported or disputed by others. Note that the citation numbers reported in this post may quickly become outdated as scite continues ingesting millions of new articles.

Overall, we collect 183 articles that have received a total of 537 supporting citations and 61 disputing citations, giving the Ig Nobel Prize papers a scite index (SI) of 0.898. The scite index (SI) is calculated as:

SI= (number of supporting citation statements) / (total number of supporting + disputing citation statements)

The most-cited paper is the famous 1999 paper by Dunning and Kruger, “Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments”, for which they were awarded the Ig Nobel Prize in Psychology in 2000. To date, it has received 107 supporting citations and 13 disputing citations.

And the top 10 most supported papers are:

These 10 articles account for ⅔ of the total number of supporting citations of all Ig Nobel papers that we have collected.

The top 10 most disputed articles, which account for 79% of all disputing citation statements, are below:

By award category

Are there differences in types of citations across disciplines?

Papers in psychology, medicine, and physics have received the most supporting citation statements. These large numbers of supporting citation statements come primarily from outlier papers like Dunning and Kruger (1999) in Psychology and Maguire et al. (2000) in Medicine. These two papers account for 66% and 75%, respectively, of the supporting citation statements in their award categories, as well as 65% and 61% of the disputing citation statements in their categories.

The same is true for Economics, where 87% of the supporting citation statements come from one paper, “Relationship of Stress, Distress, and Inadequate Coping Behaviors to Periodontal Disease” by Genco et al. (1999).

Notice the different axes of the supporting and disputing figures: supporting citations are much more common than disputing citations.

Generally, highly-cited papers tend to have a relatively high number of both supporting and disputing citation statements. This is not too surprising; more people are talking about these papers, and highly-cited papers may attract researchers to focus on supporting or disputing the claims of a paper. This is where the scite index (SI) is useful, as it corrects for the volume of citation statements by converting to a proportion SI ∈ [0,1].

When we examine SI instead of tallies, we find that disciplines become more comparable to one another. Biology, literature, and medicine have the lowest SI, while economics, physics, and chemistry have the highest. Of course, different citation practices across different disciplines may create inherent differences in SI that are not explained by the reliability of the research itself.

While it may be difficult to directly compare SI with a small sample size (the average number of papers in each of these 7 categories is just 17), a good rule-of-thumb is that a lower SI corresponds to more controversial findings — a lower SI implies a relatively high number of disputing citations, which may serve as an indicator that an article had some flaws or simply that others have failed to support its findings.

By year

How have citations to IG Nobel articles changed over the years?

Again, the large spike in citation statements that we see for papers published in 1999 and 2000 comes primarily from Dunning and Kruger (1999) and Maguire et al. (2000).

We can see that, in addition to occurring in lower numbers, disputing citation statements also occur less frequently. There are a few explanations for this.

Previous research has demonstrated that there is a strong positive-result bias in scientific publication, which is most prevalent in biological/medical science and least prevalent in natural science such as physics and chemistry. This could be one contributing factor to why disputing citations occur in significantly lower numbers.

There are many other reasons why we almost always observe fewer disputing citations than supporting citations, however. For example, it could be an indicator that the peer-review process is functioning as it should. Perhaps papers that would have been highly disputed are successfully filtered out from publication.

Conclusion

As scite continues to process millions of new citation statements, these numbers may quickly become outdated. But as of today, of the 183 Ig Nobel Prize-winning papers collected from Wikipedia, 173 have been cited at least one time. Only 82 have received supporting citation statements and 23 have received disputing citation statements.

Of the 22 papers that have received both supporting and disputing citation statements, there are a handful of papers that receive the vast majority of all citations in the dataset. The Top 5 most-cited papers receive 76% of all supporting citation statements and 58% of all disputing citation statements. If we consider the Top 10 most-cited papers, these numbers become 90% and 72%, respectively.

scite Index (SI) of IG Nobel Papers varies by discipline and across years: chemistry, economics, and physics have the highest SI values, while biology and literature have the lowest.

Want to check for yourself? You can take a look at the Custom Dashboard we created to view and download the citation data we have on all 183 papers.