No abstract

Abstract-Researchers with large-scale data-intensive applications often wish to scale up applications to run on multiple clusters, employing a middleware layer for resource management across clusters. However, at the very largest scales, such middleware is often "unfriendly" to individual clusters, which are usually designed to support communication within the cluster, not outside of it. To address this problem we have modified the Work Queue master-worker application framework to support a hierarchical configuration that more closely matches the physical architecture of existing clusters. Using a synthetic application we explore the properties of the system and evaluate its performance under multiple configurations, with varying worker reliability, network capabilities, and data requirements. We show that by matching the software and hardware architectures more closely we can gain both a modest improvement in runtime and a dramatic reduction in network footprint at the master. We then run a scalable molecular dynamics application (AWE) to examine the impact of hierarchy on performance, cost and efficiency for real scientific applications and see a 96% reduction in network footprint, making it much more palatable to system operators and opening the possibility of increasing the application scale by another order of magnitude or more.

New scintillators and waveshifter materials are under development for use in detecting charged particles in tracking applications and for detecting showering particles in calorimetric applications. Goals have been to identify and produce fast and efficient dye materials that fluoresce in the middle of the visible spectrum where polystyrene and polyvinyl toluene have good optical transparency, to replace existing materials currently in use in the field of particle physics. As a result of this study, several fluorescent dyes have been identified with fast and efficient emission, that fluorescence in the green (λ ~ 490-520 nm). These are candidate materials for new scintillators and waveshifters.

Over the last few decades, computing performance, memory capacity, and disk storage have all increased by many orders of magnitude. However, I/O performance has not increased at nearly the same pace: a disk arm movement is still measured in milliseconds, and disk I/O throughput is still measured in megabytes per second. If one wishes to build computer systems that can store and process petabytes of data, they must have large numbers of disks and the corresponding I/O paths and memory capacity to support the desired data rate. A cost efficient way to accomplish this is by clustering large numbers of commodity machines together. This chapter presents Chirp as a building block for clustered data intensive scientific computing. Chirp was originally designed as a lightweight file server for grid computing and was used as a “personal” file server. The authors explore building systems with very high I/O capacity using commodity storage devices by tying together multiple Chirp servers. Several real-life applications such as the GRAND Data Analysis Grid, the Biometrics Research Grid, and the Biocompute Facility use Chirp as their fundamental building block, but provide different services and interfaces appropriate to their target communities.

No abstract