Reliance on Science by Inventors: Hybrid Extraction of In-text Patent-to-Article Citations

Abstract

We curate and characterize a complete set of citations from patents to scientific articles, including nearly 16 million from the full text of USPTO and EPO patents. Combining heuristics and machine learning, we achieve 25% higher performance than machine learning alone. At 99.4% accuracy, coverage of 87.6% is achieved, and coverage above 90% with accuracy above 93%. Performance is evaluated with a set of 5,939 randomly-sampled, cross-verified "known good" citations, which the authors have never seen. We compare these "in-text" citations with the "official" citations on the front page of patents. In-text citations are more diverse temporally, geographically, and topically. They are less self-referential and less likely to be recycled from one patent to the next. That said, in-text citations have been overshadowed by front-page in the past few decades, dropping from 80% of all paper-to-patent citations to less than 40%. In replicating two published articles that use only citations on the front page of patents, we show that failing to capture those in the body text leads to understating the relationship between academic science and commercial invention. All patent-to-article citations, as well as the knowngood test set, are available at http://relianceonscience.org.

Knowledge for a warmer world: a patent analysis of climate change adaptation technologies

“…We also collect 408,348 patents related to climate change mitigation in buildings, carbon capture and storage (CCS), energy-saving information and communication technology (ICT), energy, clean production, transport and waste (see Veefkind et al, 2012;Angelucci et al, 2018). 2 We supplement the data with data on patent citations and co-classifications (Hötte et al, 2021a), science citations (Marx and Fuegi, 2019;Marx and Fuegi, 2020c;Marx and Fuegi, 2020b) and public R&D support (Fleming et al, 2019b;Fleming et al, 2019a) (see Sec. 4.1).…”

“…To analyze the technological and scientific knowledge base, we use data on (1) citations from patents to patents from Pichler et al, 2020 and (2) citations from patents to science provided by Marx and Fuegi, 2020b. For a description of these data and the use of science citations see Hötte et al, 2021b andFuegi, 2019.…”

“…The matching procedure is probabilistic. Marx and Fuegi (2020c) tagged citation links with a so-called confidence score, which indicates the precision and recall rate of the matching (more detail available in Marx and Fuegi, 2019;Marx and Fuegi, 2020a). We only include patents with a confidence score > 4 which is associated with a precision rate of more than 99%.…”

Innovation: Market Failures and Public Policies

“…Further, the text of patents themselves often contains information written by inventors about the precursors of their invention. Standardized datasets of scientific references in patent text now exist (Bryan, Ozcan, and Sampat, 2020;Marx and Fuegi, 2020), but more complex uses of natural language data to study technology diffusion are underexploited.…”

The Changing Structure of American Innovation: Some Cautionary Remarks for Economic Growth

“…Not surprisingly, the "Google translate" paper has 31 coauthors, many of them leading researchers in their respective fields (Wu et al, 2016). This seems to be a broader trend separating university research from industry research in this area: using data from Marx (2019), we examined the average number of coauthors in the five leading machine learning conferences in Hartmann and Henkel (2019) from 2011 to 2018 and found that research by large firms features on average one more co-author (4.3) than non-large firm papers (3.4). 16 These firms make up 10 percent (2,168 out of 20,989) of the papers published with fewer than eleven authors, but comprise 28 percent (22 out of 79) of the papers published with more than eleven authors.…”