funcX: A Federated Function Serving Fabric for Science

Chard, Ryan; Babuji, Yadu; Li, Zhuozhao; Skluzacek, Tyler J.; Woodard, A.; Blaiszik, Ben; Foster, Ian; Chard, Kyle

doi:10.1145/3369583.3392683

Cited by 153 publications

(56 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first phase of the pipeline is integrated with the Advanced Photon Source (APS) Data Management System at the beamline, which deposits each newly acquired image into a Globus-accessible storage system at the APS. As new images are acquired, Globus Automate flows are launched to process them as follows: 1) moves new files from APS to Theta by using the Globus Transfer service ( 35 ); 2) performs DIALS stills_process ( 36 ) on batches of 256 images by using funcX ( 37 ), a function-as-a-service computation system [funcX uses Parsl ( 38 ) to abstract and acquire nodes on Theta as needed, and dispatches tasks to available nodes]; 3) extracts metadata from files regarding identified diffractions and generates visualizations (funcX) showing the locations of positive hits on the mesh; and 4) publishes raw data, metadata, and visualizations to a portal on the Argonne Leadership Computing Facility (ALCF) Petrel data system ( 39 ). The result of this automated process is an indexed, searchable data collection that provides full traceability from data acquisition to processed data that can be used to inspect and update the running experiment.…”

Section: Methodsmentioning

confidence: 99%

2′-O methylation of RNA cap in SARS-CoV-2 captured by serial crystallography

Wilamowski

Sherrell

Minasov

et al. 2021

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

The genome of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) coronavirus has a capping modification at the 5′-untranslated region (UTR) to prevent its degradation by host nucleases. These modifications are performed by the Nsp10/14 and Nsp10/16 heterodimers using S-adenosylmethionine as the methyl donor. Nsp10/16 heterodimer is responsible for the methylation at the ribose 2′-O position of the first nucleotide. To investigate the conformational changes of the complex during 2′-O methyltransferase activity, we used a fixed-target serial synchrotron crystallography method at room temperature. We determined crystal structures of Nsp10/16 with substrates and products that revealed the states before and after methylation, occurring within the crystals during the experiments. Here we report the crystal structure of Nsp10/16 in complex with Cap-1 analog (m7GpppAm2′-O). Inhibition of Nsp16 activity may reduce viral proliferation, making this protein an attractive drug target.

show abstract

Section: Methodsmentioning

confidence: 99%

2′-O methylation of RNA cap in SARS-CoV-2 captured by serial crystallography

Wilamowski

Sherrell

Minasov

et al. 2021

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…These services are complementary, as they can be integrated into SCIFFS. Finally, we note works that demonstrate the benefits of serverless for data analytics (e.g., PyWren [29], IBM-PyWren [46], Flint [30], Locus [43], and funcX [19]).…”

Section: Related Workmentioning

confidence: 99%

SCIFFS: Enabling Secure Third-Party Security Analytics using Serverless Computing

Polinsky

Datta

Bates

et al. 2021

Proceedings of the 26th ACM Symposium on Access Control Models and Technologies

View full text Add to dashboard Cite

Third-party security analytics allow companies to outsource threat monitoring tasks to teams of experts and avoid the costs of inhouse security operations centers. By analyzing telemetry data from many clients these services are able to offer enhanced insights, identifying global trends and spotting threats before they reach most customers. Unfortunately, the aggregation that drives these insights simultaneously risks exposing sensitive client data if it is not properly sanitized and tracked.In this work, we present SCIFFS, an automated information flow monitoring framework for preventing sensitive data exposure in third-party security analytics platforms. SCIFFS performs decentralized information flow control over customer data in a serverless setting, leveraging the innate polyinstantiated nature of serverless functions to assure precise and lightweight tracking of data flows. Evaluating SCIFFS against a proof-of-concept security analytics framework on the widely-used OpenFaaS platform, we demonstrate that our solution supports common analyst workflows (data ingestion, custom dashboards, threat hunting) while imposing just 3.87% runtime overhead on event ingestion and the overhead on aggregation queries grows linearly with the number of records in the database (e.g., 18.75% for 50,000 records and 104.27% for 500,000 records) as compared to an insecure baseline. Thus, SCIFFS not only establishes a privacy-respecting model for third-party security analytics, but also highlights the opportunities for security-sensitive applications in the serverless computing model. CCS Concepts• Security and privacy → Information flow control; Access control; Distributed systems security.

show abstract

“…While most previous work has focused on execution of serverless functions on homogeneous systems, Chard et al [4] propose funcX, a distributed function execution platform that supports various cloud platforms and modern HPC systems with underlying heterogeneous compute nodes. In contrast to our work, funcX does not support synchronous training of ML models, and is specifically designed for scientific computing.…”

Section: Related Workmentioning

confidence: 99%

“…These range from resource-constrained edge devices to modestly priced servers with mid-range resources to expensive highperformance computers with extensive compute, storage, and network capabilities. The introduction of serverless computing, particularly function-as-a-service (FaaS) [2] has made the execution of functions on these heterogeneous devices along with the cloud possible [4].…”

mentioning

confidence: 99%