Managing scientific data

Ailamaki, Anastasia; Kantere, Verena; Dash, Debabrata

doi:10.1145/1743546.1743568

Cited by 79 publications

(55 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using vocabularies in 69 vocabulary governance groups to request new concepts, rather than attempting to add new 75 concepts to existing vocabularies at the end of a project, when timescales for completion of 76 work are often compressed, and may be insufficient to allow for requests to external 77 agencies to be processed. Once the desired vocabularies have been selected, development 78 of automated methods for tagging datasets provides the advantages of minimising the time 79 required for tagging and increasing the accuracy with which it is carried out, since it reduces 80 human error (Ailamaki et al 2010). Data can also be tagged at point of source i.e.…”

Section: Sector 59 60mentioning

confidence: 99%

Automated tagging of environmental data using a novel SKOS formatted environmental thesaurus

2014

View full text Add to dashboard Cite

There is increasing need to use the widest range of data to address issues of environmental 19 management and change, which is reflected in increasing emphasis from government 20 funding agencies for better management and access to environmental data. Bringing 21 together different environmental datasets to confidently enable integrated analysis requires 22 reference to common standards and definitions, which are frequently lacking in 23 environmental data, due to the broad subject area and lack of metadata. Automatic 24 inclusion within datasets of controlled vocabulary concepts from publicly available standard 25 vocabularies facilitates accurate annotation and promotes efficiency of metadata creation. 26To this end, we have developed a thesaurus capable of describing environmental chemistry 27 datasets. We demonstrate a novel method for tagging datasets, via insertion of this 28 thesaurus into a Laboratory Information Management System, enabling automated tagging 29 of data, thus promoting semantic interoperability between tagged data resources. Being web 30 available, and formatted using the Simple Knowledge Organisation System (SKOS) 31 semantic standard, this thesaurus is capable of providing links both to and from other 32 relevant thesauri, thus facilitating a linked data approach. Future developments will see 33 extension of the thesaurus by the user community, in terms of both concepts included and 34 links to externally hosted vocabularies. By employing a Linked Open Data approach, we 35 anticipate that Web-based tools will be able to use concepts from the thesaurus to discover 36 and link data to other information sources, including use in national assessment of the extent 37 and condition of environmental resources. by Tim Berners-Lee, refers to a set of best practices for publishing and connecting structured 48 data on the web. The concept of standardised web-accessible links within data, or 49 documents, can be used to address issues of interoperability, within the field of 50 environmental data, as described here, or any other discipline. The creation of a 'world wide 51 web of data' whereby pieces of data and information are semantically related to other 52 relevant information can greatly enhance the user's ability to derive additional value about a 53 concept with little extra effort (Bizer et al. 2009), and can facilitate interoperability both within 54 and across domains. Here, we describe the preliminary steps to implementing these 55 concepts within a national environmental chemistry analysis facility, funded by the UK 56 Natural Environment Research Council (NERC), though the steps and issues discussed will 57 be relevant to any area of research wishing to promote interoperability and re-use in their 58 sector. 59 60There is increasing pressure on publicly funded research institutes to demonstrate value for 61 money in the data they produce and enable others to re-use and add value to these data 62 (e.g. Research Councils UK (RCUK) common principles on data policy). However, data 63 lacki...

show abstract

Section: Sector 59 60mentioning

confidence: 99%

Automated tagging of environmental data using a novel SKOS formatted environmental thesaurus

2014

View full text Add to dashboard Cite

show abstract

“…Editing and updating of data also generates data. Produced data are schema-less, semi or fully structured persisting in different repositories [5]. According to some sources [2], the data volumes are approximately doubling each year.…”

Section: Introductionmentioning

confidence: 99%

Towards a Service-Based Framework for Environmental Data Processing

Madjarov¹,

Kocijan²,

Grancharova³

et al. 2014

IJACSA

View full text Add to dashboard Cite

Abstract-Scientists are confronted with significant data management problems due to the huge volume and high complexity of environmental data. An important aspect of environmental data management is that data, needed for a process, are not always in the adequate format. In this contribution, we analyze environmental data structure, and model this data using a semantic-based method. Using this model, we design and implement a framework based on Web services for transformation between massive environmental text-based data and relational databases. We present a mapping model for environmental data transformation to be used in the scenario devoted to the methodology for development of stochastic models for prediction of environmental parameters by application of Gaussian processes.

show abstract

“…The major software companies, such as Google, Amazon, Microsoft or Facebook have adapted their architectures in order to support the enormous quantity of information that they have to manage. Scientific applications are also struggling with those kinds of scenarios and significant research efforts are directed to deal with it [4]. An example of these applications is the management of astronomical catalogs; for instance those generated by the Dark Energy Survey (DES) [1] project with which we are collaborating.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Workload-Based Partitioning Algorithms for Continuously Growing Databases

Liroz-Gistau

Akbarinia

Pacitti

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Applications with very large databases, where data items are continuously appended, are becoming more and more common. Thus, the development of efficient data partitioning is one of the main requirements to yield good performance. In the case of applications that have complex access patterns, e.g. scientific applications, workload-based partitioning could be exploited. However, existing workload-based approaches, which work in a static way, cannot be applied to very large databases. In this paper, we propose DynPart and DynPartGroup, two dynamic partitioning algorithms for continuously growing databases. These algorithms efficiently adapt the data partitioning to the arrival of new data elements by taking into account the affinity of new data with queries and fragments. In contrast to existing static approaches, our approach offers constant execution time, no matter the size of the database, while obtaining very good partitioning efficiency. We validated our solution through experimentation over real-world data; the results show its effectiveness.

show abstract

Managing scientific data

Cited by 79 publications

References 18 publications

Automated tagging of environmental data using a novel SKOS formatted environmental thesaurus

Automated tagging of environmental data using a novel SKOS formatted environmental thesaurus

Towards a Service-Based Framework for Environmental Data Processing

Dynamic Workload-Based Partitioning Algorithms for Continuously Growing Databases

Contact Info

Product

Resources

About