A new data science research program: evaluation, metrology, standards, and community outreach

Dorr, Bonnie J.; Greenberg, Craig S.; Fontana, Peter; Przybocki, Mark A.; Bras, Marion Le; Ploehn, Cathryn A.; Aulov, Oleg; Michel, Martial; Golden, Edmond J.; Chang, Wo L.

doi:10.1007/s41060-016-0016-z

Cited by 7 publications

(8 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is the motivation behind a new NIST effort focused on algorithm transferability where the goal is to allow algorithms developed in one field to be applied to similar problems in other disciplines. The next iteration of the NIST DSE Series (Dorr et al, 2016a, 2016b) will combine sets of related tasks from different domains to help drive this idea of algorithm transferability forward. Accomplishing this requires standardizing data formats to allow integration into a central automatic-scoring system.…”

Section: Discussionmentioning

confidence: 99%

“…As a part of the early stages of DSE, a pilot evaluation was run using traffic data, which was then followed by this competition on converting remote sensing data to information on trees. As a component of this endeavor, NIST researchers identified general classes of data science problems (Dorr et al, 2015, 2016a, 2016b; Greenberg et al, 2014) and produced a framework for evaluating methods both within an individual domain (like in this paper) and across domains (e.g., allowing algorithms for similar tasks to be applied to both traffic and ecological problems). The NIST DSE platform was used as the foundation for the NIST DSE Plant Identification with NEON Remote Sensing Data () DSE.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A data science challenge for converting airborne remote sensing data into ecological information

Marconi

Graves

Gong

et al. 2019

PeerJ

Self Cite

View full text Add to dashboard Cite

Ecology has reached the point where data science competitions, in which multiple groups solve the same problem using the same data by different methods, will be productive for advancing quantitative methods for tasks such as species identification from remote sensing images. We ran a competition to help improve three tasks that are central to converting images into information on individual trees: (1) crown segmentation, for identifying the location and size of individual trees; (2) alignment, to match ground truthed trees with remote sensing; and (3) species classification of individual trees. Six teams (composed of 16 individual participants) submitted predictions for one or more tasks. The crown segmentation task proved to be the most challenging, with the highest-performing algorithm yielding only 34% overlap between remotely sensed crowns and the ground truthed trees. However, most algorithms performed better on large trees. For the alignment task, an algorithm based on minimizing the difference, in terms of both position and tree size, between ground truthed and remotely sensed crowns yielded a perfect alignment. In hindsight, this task was over simplified by only including targeted trees instead of all possible remotely sensed crowns. Several algorithms performed well for species classification, with the highest-performing algorithm correctly classifying 92% of individuals and performing well on both common and rare species. Comparisons of results across algorithms provided a number of insights for improving the overall accuracy in extracting ecological information from remote sensing. Our experience suggests that this kind of competition can benefit methods development in ecology and biology more broadly.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A data science challenge for converting airborne remote sensing data into ecological information

Marconi

Graves

Gong

et al. 2019

PeerJ

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many other challenges can benefit from solutions based on data science methodologies. For instance, some problem classes in data science are pattern detection, anomaly detection, cleaning, alignment, classification, regression, knowledge base construction, and density estimation [7]. These classes of problems are explored next.…”

Section: Solving Problems With Data Sciencementioning

confidence: 99%

“…Frequently, this process requires the use of cleaning and alignment methods to standardize data. There are a broad group of knowledge bases on the Internet, for example, Kaggle, 5 UCI Repository, 6 Quandl, 7 and MSCOCO. 8…”

Section: Knowledge Base Constructionmentioning

confidence: 99%

See 1 more Smart Citation

Looking at Data Science through the Lens of Scheduling and Load Balancing

Silveira¹,

Reis²,

Bavaresco³

et al. 2020

Scheduling Problems - New Applications and Trends

View full text Add to dashboard Cite

The growth in data generated by private and public organizations leads to several opportunities to obtain valuable knowledge. In this scenario, data science becomes pertinent to define a structured methodology to extract valuable knowledge from raw data. It encompasses a heterogeneous group of techniques that challenge the implementation of a single platform capable of incorporating all the available resources. Thus, it is necessary to formulate a data science workflow based on different tools to extract knowledge from massive datasets. In this context, highperformance computing (HPC) provides the infrastructure required to optimize the processing time of data science workflows, which become a collection of tasks that must be efficiently scheduled to provide results in acceptable time intervals. While few studies explore the use of HPC for data science tasks, in the best of our knowledge, none conducts an in-depth analysis of scheduling and load balancing on such workflows. In this context, this chapter proposes an analysis of scheduling and load balancing from the perspective of data science scenarios. It presents concepts, environments, and tools to summarize the theoretical background required to define, assign, and execute data science workflows. Furthermore, we are also presenting new trends concerning the intersection of data science, scheduling, and load balance.

show abstract

Data Science with Semantic Technologies: Application to Information Systems Development

Sassi

Yanes

2023

Journal of Computer Information Systems

View full text Add to dashboard Cite

A new data science research program: evaluation, metrology, standards, and community outreach

Cited by 7 publications

References 43 publications

A data science challenge for converting airborne remote sensing data into ecological information

A data science challenge for converting airborne remote sensing data into ecological information

Looking at Data Science through the Lens of Scheduling and Load Balancing

Data Science with Semantic Technologies: Application to Information Systems Development

Contact Info

Product

Resources

About