Know thy tools! Limits of popular algorithms used for topic reconstruction

Abstract

For the reconstruction of topics in bibliometric networks, one has to use algorithms. Specifically, researchers often apply algorithms from the class of network community detection algorithms (such as the Louvain algorithm), which are general-purpose algorithms not intentionally programmed for a bibliometric task. Each algorithm has specific properties “inscribed”, which distinguishes it from the others. It can thus be assumed that different algorithms are more or less suitable for a given bibliometric task. However, the suitability of a specific algorithm when it is applied for topic reconstruction is rarely reflected upon. Why choose this algorithm and not another? In this study I assess the suitability of four community detection algorithms for topic reconstruction, by first deriving the properties of the phenomenon to be reconstructed – topics – and comparing if these match with the properties of the algorithms. The results suggest that the previous use of these algorithms for bibliometric purposes cannot be justified by their specific suitability for this task. Peer Review https://publons.com/publon/10.1162/qss_a_00217

Mapping the field of cultural evolutionary theory and methods in archaeology using bibliometric methods

“…Rahiminejad et al, 2019). Like other community detection/clustering algorithms it does, though, require careful analysis of the obtained results and their limitations (Held, 2022).…”

From Effects of Governance to Causes of Epistemic Change

“…Changes in the distribution of effort across topics in a scientific community are difficult to identify because the knowledge a scientific community works with must be delineated, topics be identified, and efforts on topics measured. Unfortunately, bibliometrics has not yet developed robust methods for delineating scientific communities and their knowledge (Held et al 2021;Held 2022). International scientific communities and their topics are units of analysis that cannot currently be delineated with the necessary validity and reliability.…”

A local cohesion-maximising algorithm for the exploration of publication networks

“…Thematic similarity translates into above-average subgraph density in bibliographic coupling networks. The insider perspective translates into the use of local information (information about a subgraph and its environment) for its delineation (Held, 2022). Taken together, these translations suggest experimenting with local density-maximising algorithms, which can be applied to traditional bibliometric data models like direct citation and bibliographic coupling.…”

“…The dominant approach applies global algorithmsalgorithms that partition the whole network by optimising a global quality function with data models based on direct citation or bibliographic coupling and interprets the resulting clusters as topics (Gläser et al, 2017). The most popular global algorithms prioritise the separation of clusters over their coherence (Held, 2022). This approach is problematic because it inadvertently decouples the bibliometric reconstruction of topics from the sociological discussion of their role in the production of scientific knowledge.…”