A heuristic to optimize the execution cost of distributed seismic processing programs on the cloud

Okita, Nicholas T.; Coimbra, Tiago A.; Tygel, Martin; Borin, Edson

doi:10.1190/segam2019-3216462.1

Cited by 8 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Okita et al . (2019) implement a heuristic to determine the best instances set to use on the cloud by initiating and terminating multiple nodes, which could lead to a behaviour similar to what we see in the four nodes failure column in Figure 11. In such a situation, IC could become very helpful.…”

Section: Discussionmentioning

confidence: 94%

“…Notably, even without changing the programming model framework itself, the IC technique can be applied when we terminate multiple nodes in a short period. For example, Okita et al (2019) implement a heuristic to determine the best instances set to use on the cloud by initiating and terminating multiple nodes, which could lead to a behaviour similar to what we see in the four nodes failure column in Figure 11. In such a situation, IC could become very helpful.…”

Section: Independent Checkpointingmentioning

confidence: 96%

See 1 more Smart Citation

High‐performance computing strategies for seismic‐imaging software on the cluster and cloud‐computing environments

Okita

Camargo

Ribeiro

et al. 2021

Geophysical Prospecting

Self Cite

View full text Add to dashboard Cite

Modern subsurface imaging techniques allow obtaining high‐quality images but with high computational costs. Nonetheless, depending on the amount of data, their execution is limited by memory in the current generation's hardware. However, with the advancement of new hardware and cloud‐based solutions, these problems are mitigated but still with the risk of work loss and instability. To mitigate the execution problems in memory‐limited and fail‐prone environments, we propose two high‐performance computing techniques. The first is based on independent checkpointing alongside a fault‐tolerant framework to store an execution state and recover from that state in case of failures. Besides, for memory‐limited graphics processing units, we present a technique to reduce the amount of memory requirement that we call the hybrid strategy. The experiments showed that the independent checkpointing alongside the fault‐tolerant framework is able to mitigate the performance penalty of node failures, with the independent checkpointing technique being more relevant when multiple nodes are terminated. Furthermore, the hybrid strategy technique has shown the possibility of execution of larger models that could make the graphics processing unit run out of memory otherwise. Finally, our implementation is scalable, allowing a significant improvement in performance when adding new nodes. In conclusion, our techniques can be used to deliver fast, high‐fidelity subsurface imaging in unstable and memory‐limited environments, such as the cloud.

show abstract

Section: Discussionmentioning

confidence: 94%

Section: Independent Checkpointingmentioning

confidence: 96%

High‐performance computing strategies for seismic‐imaging software on the cluster and cloud‐computing environments

Okita

Camargo

Ribeiro

et al. 2021

Geophysical Prospecting

Self Cite

View full text Add to dashboard Cite

show abstract

“…Considering the trend of using cloud computing for processing of seismic data, 8‐11 all experiments took place on virtual machines of the Amazon Web Services (AWS) cloud provider. Table 1 shows the instances used, their costs and their characteristics.…”

Section: Performance Analysis Of the Main File Formats In Seismologymentioning

confidence: 99%

“…The increasing use of cloud computing in seismology 8‐11 also introduces new aspects to be considered. The cloud offers many different storage mechanisms, each one presenting different behaviors, performance characteristics and costs.…”

Section: Introductionmentioning

confidence: 99%

High‐performance IO for seismic processing on the cloud

Guimarães

Lacalle

Rodamilans

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Most of the applications in the seismology field rely on the processing of up to hundreds of terabytes of data and their performance is strongly affected by IO operations. In this article, we analyze the main file structures currently used to store seismic data and propose a new intermediate data structure to improve IO performance while still complying with established standards. We show that, throughout a common workflow in seismic data analysis, our IO performance gain greatly surpasses the overhead of translating data to the intermediate structure. This approach enables a speedup of up to 208 times in reading time when using classical standards (e.g., SEG‐Y) and our intermediate structure is up to 1.8 times more efficient than modern formats (e.g., ASDF). Considering cache‐friendly applications, our speedups over the direct use of SEG‐Y reach 8000 times. We also performed a cost analysis on the AWS cloud showing that, in our approach, HDDs can be 1.25 times more cost‐effective than SSDs.

show abstract

Deploying and Configuring Infrastructure

Borin

Napoli

2023

High Performance Computing in Clouds

View full text Add to dashboard Cite

A heuristic to optimize the execution cost of distributed seismic processing programs on the cloud

Cited by 8 publications

References 7 publications

High‐performance computing strategies for seismic‐imaging software on the cluster and cloud‐computing environments

High‐performance computing strategies for seismic‐imaging software on the cluster and cloud‐computing environments

High‐performance IO for seismic processing on the cloud

Deploying and Configuring Infrastructure

Contact Info

Product

Resources

About