Why good data analysts need to be critical synthesists. Determining the role of semantics in data analysis

Scheider, Simon; Ostermann, Frank O.; Adams, Benjamin

doi:10.1016/j.future.2017.02.046

Cited by 31 publications

(35 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While some of the publications focus on specific aspects such as data (Gewin, 2016), code (Stodden & Miguez, 2014), workflow semantics (Scheider, Ostermann & Adams, 2017), and results (Sandve et al, 2013), others provide an all-embracing set of research instructions (Stodden et al, 2016; Nosek et al, 2015; Gil et al, 2016). …”

Section: Methodsmentioning

confidence: 99%

Reproducible research and GIScience: an evaluation using AGILE conference papers

Nüst

Granell

Hofer

et al. 2018

PeerJ

Self Cite

View full text Add to dashboard Cite

The demand for reproducible research is on the rise in disciplines concerned with data analysis and computational methods. Therefore, we reviewed current recommendations for reproducible research and translated them into criteria for assessing the reproducibility of articles in the field of geographic information science (GIScience). Using this criteria, we assessed a sample of GIScience studies from the Association of Geographic Information Laboratories in Europe (AGILE) conference series, and we collected feedback about the assessment from the study authors. Results from the author feedback indicate that although authors support the concept of performing reproducible research, the incentives for doing this in practice are too small. Therefore, we propose concrete actions for individual researchers and the GIScience conference series to improve transparency and reproducibility. For example, to support researchers in producing reproducible work, the GIScience conference series could offer awards and paper badges, provide author guidelines for computational research, and publish articles in Open Access formats.

show abstract

Section: Methodsmentioning

confidence: 99%

Reproducible research and GIScience: an evaluation using AGILE conference papers

Nüst

Granell

Hofer

et al. 2018

PeerJ

Self Cite

View full text Add to dashboard Cite

show abstract

“…We call this problem geo-analytical QA, which is part of a more general endeavour of indirect QA (Scheider, Ostermann, and Adams 2017). 'Indirect' means here that answers cannot be directly filtered out by queries, but need to involve transformations first.…”

Section: Motivationmentioning

confidence: 99%

“…This latter task requires capturing the analytic potential of tools and data to answer a question. In effect, this means to translate a spatial question into a query over a transformation: The query should match 'potential' datasets generated by some workflow (Figure 2), a novel computational challenge which was called 'indirect QA' in Scheider, Ostermann, and Adams (2017).…”

Section: Challenges In Asking and Answering Geo-analytical Questionsmentioning

confidence: 99%

Geo-analytical question-answering with GIS

Scheider

Nyamsuren

Kruiger

et al. 2020

International Journal of Digital Earth

Self Cite

View full text Add to dashboard Cite

Question Answering (QA), the process of computing valid answers to questions formulated in natural language, has recently gained attention in both industry and academia. Translating this idea to the realm of geographic information systems (GIS) may open new opportunities for data scientists. In theory, analysts may simply ask spatial questions to exploit diverse geographic information resources, without a need to know how GIS tools and geodata sets interoperate. In this outlook article, we investigate the scientific challenges of geo-analytical question answering, introducing the problems of unknown answers and indirect QA. Furthermore, we argue why core concepts of spatial information play an important role in addressing this challenge, enabling us to describe analytic potentials, and to compose spatial questions and workflows for generating answers. ARTICLE HISTORY

show abstract

“…algorithms, parameters, and source code; results include (intermediate) data and parameters as well as outcomes such as statistics, maps, figures, or new datasets; and structure considers the organization and integration of the other aspects. While some of the publications focus on specific aspects such as data (Gewin, 2016), code (Stodden and Miguez, 2014), workflow semantics (Scheider et al, 2017), and results (Sandve et al, 2013), others provide an all-embracing set of research instructions Nosek et al, 2015;Gil et al, 2016).…”

Section: Recommendations and Suggestions In Literaturementioning

confidence: 99%

Publishing Reproducible Geoscientific Papers: Status quo, benefits, and opportunities

Konkol¹

2019

Preprint

View full text Add to dashboard Cite

Open reproducible research (ORR) is the practice of publishing the source code andthe datasets that are needed to produce the computational results reported in a paper.Since geoscientific articles often include geostatistical analyses and spatiotemporaldata, reproducibility should be a cornerstone of the computational geosciences butis rarely realized. In addition, the current way of publishing scientific outcomes, i.e.as static PDFs, does not adequately report on computational aspects. Thus, readerscannot fully understand how the authors came to the conclusions and how robustthese are to changes in the analysis. Consequently, it is difficult for reviewers tofollow the analysis steps, and for other researchers to reuse existing materials. Thisdissertation has two overarching goals to tackle these issues: First, it aims at assistingauthors in adhering to ORR principles to ensure high scientific standards. The secondgoal is to reap the benefits that come with papers supplemented by code and data. Toachieve these objectives, this thesis reports on five complementary studies to collectand analyze qualitative (i.e. through interviews and a focus group) and quantitativedata (i.e. through surveys and reproducibility studies). Based on these studies, thiswork provides four key contributions: First, it identifies obstacles that preventedgeoscientists from publishing ORR. To overcome these barriers, this dissertationsuggests concrete and directly applicable strategies. One of these strategies is theexecutable research compendium (ERC) which encapsulates the paper, code, data,and the entire software environment needed to produce the computational results.Based on that, this work presents the design and implementation of an ERC-basedworkflow. It allows authors to convey their computational methods and results byalso providing interactive access to code and data, and readers to deeply investigatethe computational analysis while reading the actual article, e.g. by changing theparameters of the analysis. Finally, this work describes the concept of a binding; abinding connects those code lines and data subsets that produce a specific result, e.g.a figure or number. By also considering user interface widgets (e.g. a slider), thisapproach allows readers to interactively manipulate the parameters of the analysisto see how these changes affect the result. To conclude, the contributions include (i)a set of obstacles which prevent geoscientists from publishing ORR, (ii) conceptsand tools to overcome the identified barriers, (iii) incentives and opportunities thatcome with attached code and data, and (iv) solutions to realize the incentives whicheventually result in a higher number of open and reproducible research.

show abstract

Why good data analysts need to be critical synthesists. Determining the role of semantics in data analysis

Cited by 31 publications

References 73 publications

Reproducible research and GIScience: an evaluation using AGILE conference papers

Reproducible research and GIScience: an evaluation using AGILE conference papers

Geo-analytical question-answering with GIS

Publishing Reproducible Geoscientific Papers: Status quo, benefits, and opportunities

Contact Info

Product

Resources

About