Tapis API Development with Python: Best Practices In Scientific REST API Implementation

Cleveland, Sean B.; Jamthe, Anagha; Padhy, Smruti; Stubbs, Joe; Packard, Michale; Looney, Julia; Terry, Steve; Cardone, Richard; Dahan, Maytal; Jacobs, Gwen A.

doi:10.1145/3311790.3396647

Cited by 15 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…RT components that run on the clusters are orchestrated through the Tapis platform. 27 Tapis brings together public, private, and shared HPC, high-throughput computing (HTC), Cloud, and Big Data resources under a single, web-friendly REST API. The RT uses Tapis to configure storage systems (for controlled access to data and metadata) and execution systems (for allocating jobs and applications to specialized hardware).…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…TACC provides several high performance computing (HPC) clusters that are accessible through the SD2E portal (). RT components that run on the clusters are orchestrated through the Tapis platform . Tapis brings together public, private, and shared HPC, high-throughput computing (HTC), Cloud, and Big Data resources under a single, web-friendly REST API.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Round Trip: An Automated Pipeline for Experimental Design, Execution, and Analysis

et al. 2022

Self Cite

View full text Add to dashboard Cite

Synthetic biology is a complex discipline that involves creating detailed, purpose-built designs from genetic parts. This process is often phrased as a Design-Build-Test-Learn loop, where iterative design improvements can be made, implemented, measured, and analyzed. Automation can potentially improve both the end-to-end duration of the process and the utility of data produced by the process. One of the most important considerations for the development of effective automation and quality data is a rigorous description of implicit knowledge encoded as a formal knowledge representation. The development of knowledge representation for the process poses a number of challenges, including developing effective human−machine interfaces, protecting against and repairing user error, providing flexibility for terminological mismatches, and supporting extensibility to new experimental types. We address these challenges with the DARPA SD2 Round Trip software architecture. The Round Trip is an open architecture that automates many of the key steps in the Test and Learn phases of a Design-Build-Test-Learn loop for highthroughput laboratory science. The primary contribution of the Round Trip is to assist with and otherwise automate metadata creation, curation, standardization, and linkage with experimental data. The Round Trip's focus on metadata supports fast, automated, and replicable analysis of experiments as well as experimental situational awareness and experimental interpretability. We highlight the major software components and data representations that enable the Round Trip to speed up the design and analysis of experiments by 2 orders of magnitude over prior ad hoc methods. These contributions support a number of experimental protocols and experimental types, demonstrating the Round Trip's breadth and extensibility. We describe both an illustrative use case using the Round Trip for an on-the-loop experimental campaign and overall contributions to reducing experimental analysis time and increasing data product volume in the SD2 program.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Round Trip: An Automated Pipeline for Experimental Design, Execution, and Analysis

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although this initial implementation worked, it was less performant than CHORDS and did not match with the other Tapis API development processes or supported languages. Therefore, a new implementation in python that leverage the Tapis API development best practices 34 was undertaken.…”

Section: Methodsmentioning

confidence: 99%

Tapis v3 Streams API: Time‐series and data‐driven event support in science gateway infrastructure

Cleveland

Jamthe²,

Padhy³

et al. 2020

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

The explosion of IoT devices and sensors in recent years has led to a demand for efficiently storing, processing and analyzing time-series data. Geoscience researchers use time-series data stores such as Hydroserver, Virtual Observatory and Ecological Informatics System (VOEIS), and Cloud-Hosted Real-time Data Service (CHORDS). Many of these tools require a great deal of infrastructure to deploy and expertise to manage and scale. The Tapis framework, an NSF funded project, provides science as a service APIs to allow researchers to achieve faster scientific results, by eliminating the need to set up a complex infrastructure stack. The University of Hawai'i (UH) and Texas Advanced Computing Center (TACC) have collaborated to develop an open sourceTapis Streams API that builds on the concepts of the CHORDS time series data service to support research. This new hosted service allows storing, processing, annotating, archiving, and querying time-series data in the Tapis multi-user and multi-tenant collaborative platform. The Streams API provides a hosted production level middleware service that enables new data-driven event workflows capabilities that may be leveraged by researchers and Tapis powered science gateways for handling spatially indexed time-series datasets.

show abstract

“…VII. RELATED WORK Tapis [4] provides API's for building Science Gateways. Like Custos, Tapis Security Kernel supports OpenID Connect and OAuth for authentication and uses Vault for storing secrets.…”

Section: Custos Administrative Portalmentioning

confidence: 99%

“…Science gateways can be built using open source software such as Galaxy [2], HUBzero [3], Tapis [4], Open OnDemand [5], or Apache Airavata [6]. These systems have common security requirements, including authentication and authorization, user and group management, and resource credential management.…”

Section: Introductionmentioning

confidence: 99%

Experiences with Integrating Custos SecurityServices

Ranawaka,

Liyanage,

Baker

et al. 2021

Preprint

View full text Add to dashboard Cite

Science gateways are user-facing cyberinfrastructure that provide researchers and educators with Web-based access to scientific software, computing, and data resources. Managing user identities, accounts, and permissions are essential tasks for science gateways, and gateways likewise must manage secure connections between their middleware and remote resources. The Custos project is an effort to build open source software that can be operated as a multi-tenanted service that provides reliable implementations of common science gateway cybersecurity needs, including federated authentication, identity management, group and authorization management, and resource credential management. Custos aims further to provide integrated solutions through these capabilities, delivering end-toend support for several science gateway usage scenarios. This paper examines four deployment scenarios using Custos and associated extensions beyond previously described work. The first capability illustrated by these scenarios is the need for Custos to provide hierarchical tenant management that allows multiple gateway deployments to be federated together and also to support consolidated, hosted science gateway platform services. The second capability illustrated by these scenarios is the need to support service accounts that can support non-browser applications and agent applications that can act on behalf of users on edge resources. We illustrate how the latter can be built using Web security standards combined with Custos permission management mechanisms.

show abstract

Tapis API Development with Python: Best Practices In Scientific REST API Implementation

Cited by 15 publications

References 3 publications

Round Trip: An Automated Pipeline for Experimental Design, Execution, and Analysis

Round Trip: An Automated Pipeline for Experimental Design, Execution, and Analysis

Tapis v3 Streams API: Time‐series and data‐driven event support in science gateway infrastructure

Experiences with Integrating Custos SecurityServices

Contact Info

Product

Resources

About