Open XDMoD: A Tool for the Comprehensive Management of High-Performance Computing Resources

Palmer, Jeffrey T.; Gallo, Steven M.; Furlani, Thomas R.; Jones, Matthew D.; DeLeon, Robert L.; White, John; Simakov, Nikolay A.; Patra, Abani; Sperhac, Jeanette; Yearke, Thomas; Rathsam, Ryan; Innus, Martins; Cornelius, Cynthia D.; Browne, James C.; Barth, William L.; Evans, R. T.

doi:10.1109/mcse.2015.68

Cited by 91 publications

(33 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then turned to larger national resources to supplement our computing needs. 1 Open XDMoD [18] for Georgia State University: http://xdmod.rs.gsu.edu…”

Section: Gsu Clustermentioning

confidence: 99%

Computational Challenges and Opportunities of Simulating Cosmic Ray Showers at Global Scale

Sarajlic

et al. 2018

Proceedings of the Practice and Experience on Advanced Research Computing

View full text Add to dashboard Cite

Galactic cosmic rays are the high-energy particles that stream into our solar system from distant corners of our Galaxy and some low energy particles are from the Sun which are associated with solar flares. The Earth atmosphere serves as an ideal detector for the high energy cosmic rays which interact with the air molecule nuclei causing propagation of extensive air showers. In recent years, there are growing interests in the applications of the cosmic ray measurements which range from the space/earth weather monitoring, homeland security based on the cosmic ray muon tomography, radiation effects on health via air travel, etc. A simulation program (based on the GEANT4 software package developed at CERN) has been developed at Georgia State University for studying the cosmic ray showers in atmosphere. The results of this simulation study will provide unprecedented knowledge of the geo-position-dependent cosmic ray shower profiles and significantly enhance the applicability of the cosmic ray applications. In the paper, we present the computational challenges and the opportunities for carrying out the cosmic ray shower simulations at the global scale using various computing resources including XSEDE.

show abstract

“…We then turned to larger national resources to supplement our computing needs. 1 Open XDMoD [18] for Georgia State University: http://xdmod.rs.gsu.edu…”

Section: Gsu Clustermentioning

confidence: 99%

Computational Challenges and Opportunities of Simulating Cosmic Ray Showers at Global Scale

Sarajlic

et al. 2018

Proceedings of the Practice and Experience on Advanced Research Computing

View full text Add to dashboard Cite

show abstract

“…The AT-LAS project uses Jetstream to extend their available compute resources for particle physics simulations and data analysis, through a gateway-like workflow. The CIPRES Gateway for systematic and population biology currently allows users to run a variety of computationally intense codes on XSEDE resources, including Jetstream, through a point-and-click web interface that provides access to phylogenetic reconstruction [20], and spans the total lifetime of the Jetstream project.…”

Section: Uptake Of Jetstream As a Science Gateway Hosting Environmentmentioning

confidence: 99%

Using the Jetstream Research Cloud to Provide Science Gateway Resources

Knepper

Coulter

Pierce

et al. 2017

2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)

View full text Add to dashboard Cite

Abstract-We describe the use of the Jetstream research-cloud, a purpose-built system with the goal of supporting "long-tail" research by providing a flexible, on-demand research infrastructure, to provide scalable back-end resources for science gateways. In addition to providing cloud-like resources for on-demand science, Jetstream offers the capability to instantiate long-running clusters which support science gateways. Science gateways are web-based systems built on computational infrastructure which provide commonly-used tools to a community of users. We created a persistent cluster on the Jetstream system which is connected to the SEAGrid science gateway and provides additional compute resources for a variety of quantum chemistry calculations. We discuss the further application of toolkits provided by the Extreme Science and Engineering Discovery Environment (XSEDE) to build general-purpose clusters on the research cloud.

show abstract

“…Several tools exist for collecting and visualizing resource usage data from large scale HPC installations (e.g. Texas Advanced Computing Center TACC Stats [10], XSEDE Metrics on Demand or XDMoD [17], etc.). Such tools can produce large amounts of high dimensional resource usage data at a high temporal frequency for each computational node in the system.…”

Section: Introductionmentioning

confidence: 99%

dynamicMF: A Matrix Factorization Approach to Monitor Resource Usage in High Performance Computing Systems

Sorkunlu

Luong

Chandola

2018

2018 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

High performance computing (HPC) facilities consist of a large number of interconnected computing units (or nodes) that execute highly complex scientific simulations to support scientific research. Monitoring such facilities, in real-time, is essential to ensure that the system operates at peak efficiency. Such systems are typically monitored using a variety of measurement and log data which capture the state of the various components within the system at regular intervals of time. As modern HPC systems grow in capacity and complexity, the data produced by current resource monitoring tools is at a scale that it is no longer feasible to be visually monitored by analysts. We propose a method that transforms the multi-dimensional output of resource monitoring tools to a low dimensional representation that facilitates the understanding of the behavior of a High Performance Computing (HPC) system. The proposed method automatically extracts the low-dimensional signal in the data which can be used to track the system ef-ficiency and identify performance anomalies. The method models the resource usage data as a three dimensional tensor (capturing resource usage of all compute nodes for difference resources over time). A dynamic matrix factorization algorithm, called dynamicMF, is proposed to extract a low-dimensional temporal signal for each node, which is subsequently fed into an anomaly detector. Results on resource usage data collected from the Lonestar 4 system at the Texas Advanced Computing Center show that the identified anomalies are correlated with actual anomalous events reported in the system log messages.

show abstract

Open XDMoD: A Tool for the Comprehensive Management of High-Performance Computing Resources

Cited by 91 publications

References 7 publications

Computational Challenges and Opportunities of Simulating Cosmic Ray Showers at Global Scale

Computational Challenges and Opportunities of Simulating Cosmic Ray Showers at Global Scale

Using the Jetstream Research Cloud to Provide Science Gateway Resources

dynamicMF: A Matrix Factorization Approach to Monitor Resource Usage in High Performance Computing Systems

Contact Info

Product

Resources

About