Promoting research by reducing uncertainty in academic writing: a large-scale diachronic case study on hedging in Science research articles across 25 years

Yao, Mingxin; Ying, Weiqin; Wang, Huiyu

doi:10.1007/s11192-023-04759-6

Cited by 8 publications

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two (of four) review articles included critiques, one of which recognised specific challenges but generally reflected the positive tone of the literature (Argyrou and Agapiou, 2022), while the other offered a more sustained discussion of challenges and possible solutions (Kadhim and Abed, 2023). The overwhelmingly positive tone of these papers likely indicates a certain degree of "publication bias", where positive results are more likely to be published than negative (Brown et al, 2017;Dickersin et al, 1987;Harrison et al, 2017;Ioannidis, 2005;K€ uhberger et al, 2014;Møller and Jennions, 2001), or at the very least a reflection of the rhetorical shift in scientific research towards less qualified or uncertain presentation of outcomes (Vinkers et al, 2015;Wheeler et al, 2021;Yao et al, 2023;Yuan and Yao, 2022). In this context, it is important to document unsuccessful attempts to apply ML techniques to archaeological remote sensing, or at least to highlight problems researchers are likely to face as they adopt the technology.…”

Section: Automated Approaches To Remotely Sensed Datamentioning

confidence: 99%

Validating predictions of burial mounds with field data: the promise and reality of machine learning

Sobotkova,

Kristensen-McLachlan,

Mallon

et al. 2024

View full text Add to dashboard Cite

PurposeThis paper provides practical advice for archaeologists and heritage specialists wishing to use ML approaches to identify archaeological features in high-resolution satellite imagery (or other remotely sensed data sources). We seek to balance the disproportionately optimistic literature related to the application of ML to archaeological prospection through a discussion of limitations, challenges and other difficulties. We further seek to raise awareness among researchers of the time, effort, expertise and resources necessary to implement ML successfully, so that they can make an informed choice between ML and manual inspection approaches.Design/methodology/approachAutomated object detection has been the holy grail of archaeological remote sensing for the last two decades. Machine learning (ML) models have proven able to detect uniform features across a consistent background, but more variegated imagery remains a challenge. We set out to detect burial mounds in satellite imagery from a diverse landscape in Central Bulgaria using a pre-trained Convolutional Neural Network (CNN) plus additional but low-touch training to improve performance. Training was accomplished using MOUND/NOT MOUND cutouts, and the model assessed arbitrary tiles of the same size from the image. Results were assessed using field data.FindingsValidation of results against field data showed that self-reported success rates were misleadingly high, and that the model was misidentifying most features. Setting an identification threshold at 60% probability, and noting that we used an approach where the CNN assessed tiles of a fixed size, tile-based false negative rates were 95–96%, false positive rates were 87–95% of tagged tiles, while true positives were only 5–13%. Counterintuitively, the model provided with training data selected for highly visible mounds (rather than all mounds) performed worse. Development of the model, meanwhile, required approximately 135 person-hours of work.Research limitations/implicationsOur attempt to deploy a pre-trained CNN demonstrates the limitations of this approach when it is used to detect varied features of different sizes within a heterogeneous landscape that contains confounding natural and modern features, such as roads, forests and field boundaries. The model has detected incidental features rather than the mounds themselves, making external validation with field data an essential part of CNN workflows. Correcting the model would require refining the training data as well as adopting different approaches to model choice and execution, raising the computational requirements beyond the level of most cultural heritage practitioners.Practical implicationsImproving the pre-trained model’s performance would require considerable time and resources, on top of the time already invested. The degree of manual intervention required – particularly around the subsetting and annotation of training data – is so significant that it raises the question of whether it would be more efficient to identify all of the mounds manually, either through brute-force inspection by experts or by crowdsourcing the analysis to trained – or even untrained – volunteers. Researchers and heritage specialists seeking efficient methods for extracting features from remotely sensed data should weigh the costs and benefits of ML versus manual approaches carefully.Social implicationsOur literature review indicates that use of artificial intelligence (AI) and ML approaches to archaeological prospection have grown exponentially in the past decade, approaching adoption levels associated with “crossing the chasm” from innovators and early adopters to the majority of researchers. The literature itself, however, is overwhelmingly positive, reflecting some combination of publication bias and a rhetoric of unconditional success. This paper presents the failure of a good-faith attempt to utilise these approaches as a counterbalance and cautionary tale to potential adopters of the technology. Early-majority adopters may find ML difficult to implement effectively in real-life scenarios.Originality/valueUnlike many high-profile reports from well-funded projects, our paper represents a serious but modestly resourced attempt to apply an ML approach to archaeological remote sensing, using techniques like transfer learning that are promoted as solutions to time and cost problems associated with, e.g. annotating and manipulating training data. While the majority of articles uncritically promote ML, or only discuss how challenges were overcome, our paper investigates how – despite reasonable self-reported scores – the model failed to locate the target features when compared to field data. We also present time, expertise and resourcing requirements, a rarity in ML-for-archaeology publications.

show abstract

Section: Automated Approaches To Remotely Sensed Datamentioning

confidence: 99%

Validating predictions of burial mounds with field data: the promise and reality of machine learning

Sobotkova,

Kristensen-McLachlan,

Mallon

et al. 2024

View full text Add to dashboard Cite

show abstract

Slow Science But Fast Forward: The Political Economy of Rock Art Research in A Globalized World

Conkey

2024

Interdisciplinary Contributions to Archaeology

View full text Add to dashboard Cite

There is no doubt that the past decades have brought exciting and novel understandings about geographic distributions, chronologies and analytical methods to the studies of rock art. Even from the lurch into the twenty-first century, this has been a fast forward: increasing confirmations of early image-making in Australia and other places; successful application of a new dating method to reveal previously unimagined figurative images in very deep time in Borneo; a proliferation of rock art knowledge and research; and expanded and inter-connected communities of researchers are just a few among many examples of fast-breaking news for the field. But at the same time, some of the practices that are decried by the arena of “slow science” are still with us and have, perhaps, precisely as part of the “globalization” of rock art research, become more entrenched by those who consider the field to be more competitive than collaborative, still motivated by the pull of “origins” research and claims, and the lack of retractions when, indeed, a need for such is at hand and for the betterment of the field. Slow science promotes time to think, rather than haste to get out the big next “scoop”; it promotes the reminder that we are enmeshed more than ever in broader social interests, human experiences and human needs, and for a more lasting and even an ethical science, racing ahead is deeply problematic. This chapter will explore the issues implicated by the fast-moving world with its dampening of local knowledges and alienations of non-experts as is situated in rock art research and the benefits/mandates of what slow science can bring to the field. In fact, I will suggest that rock art research is an ideal field for advancing the benefits and the power of slow science.

show abstract

Expressions of confusion in research articles: a diachronic cross-disciplinary investigation

Wang,

2023

Scientometrics

View full text Add to dashboard Cite

Promoting research by reducing uncertainty in academic writing: a large-scale diachronic case study on hedging in Science research articles across 25 years

Cited by 8 publications

References 28 publications

Validating predictions of burial mounds with field data: the promise and reality of machine learning

Validating predictions of burial mounds with field data: the promise and reality of machine learning

Slow Science But Fast Forward: The Political Economy of Rock Art Research in A Globalized World

Expressions of confusion in research articles: a diachronic cross-disciplinary investigation

Contact Info

Product

Resources

About