We report on numerical simulations of the detailed evolution of the single mode Rayleigh-Taylor [Lord Rayleigh, Scientific Papers II (Cambridge University Press, Cambridge, 1900), p. 200; G. I. Taylor, “The instability of liquid surfaces when accelerated in a direction perpendicular to their plane,” Proc. R. Soc. London, Ser. A 201, 192 (1950)10.1098/rspa.1950.0052; S. Chandrasekhar, Hydrodynamic and Hydromagnetic Stability (Oxford University Press, Oxford, 1961)] instability to late times and high aspect ratios. In contrast to established potential flow models that predict a terminal velocity and a constant Froude number at low Atwood numbers, we observe a complex sequence of events that can be summarized in four stages: I. Exponential growth of imposed perturbations, II. Saturation to terminal velocity, III. Reacceleration to a higher Froude number, and IV. Chaotic mixing. The observed reacceleration away from the Froude number predicted by potential flow theory is attributed to the appearance of secondary Kelvin–Helmholtz structures, and described with a modification to the potential flow model proposed by Betti and Sanz [R. Betti and J. Sanz, “Bubble acceleration in the ablative Rayleigh-Taylor instability,” Phys. Rev. Lett. 97, 205002 (2006)10.1103/PhysRevLett.97.205002]. The secondary KH instability is in turn sensitive to several parameters, and can be suppressed at large Atwood numbers, as well as viscosity (physical or numerical), with the bubble/spike velocity in each case reverting to the potential flow value. Our simulations delineate the change in dynamics of the primary and secondary instabilities due to changes in these flow parameters. When the flow is allowed to evolve to late times, further instability is observed, resulting in chaotic mixing which is quantified here. The increased atomic mixing due to small-scale structures results in a dramatic drop in the late-time Froude number. Spike behavior resembles bubbles at low A, while for large A, spikes approach free-fall – thus, the notion of a terminal velocity appears not to be applicable to spikes at any density difference. We expect the results to be relevant to turbulent mix models that are based on bubble growth and interaction.

SUMMARYWe report initial experience with gas dynamics simulation on the Los Alamos Roadrunner machine. In this initial work, we have restricted our attention to flows in which the flow Mach number is less than 2. This permits us to use a simplified version of the PPM gas dynamics algorithm that has been described in detail by Woodward (2006). We follow a multifluid volume fraction using the PPB moment-conserving advection scheme, enforcing both pressure and temperature equilibrium between two monatomic ideal gases within each grid cell. The resulting gas dynamics code has been extensively restructured for efficient multicore processing and implemented for scalable parallel execution on the Roadrunner system. The code restructuring and parallel implementation are described and performance results are discussed. For a modest grid size, sustained performance of 3.89 Gflop s −1 CPU-core −1 is delivered by this code on 36 Cell processors in 9 triblade nodes of a single rack of Roadrunner hardware.

No abstract

Computational Fluid Dynamics is an important area in scientific computing. The weak scaling of codes is well understood with about two decades of experiences using MPI. As a result, per-node performance has become very crucial to the overall machine performance. However, despite the use of multi-threading, obtaining good performance at each core is still extremely challenging. The challenges are primarily due to memory bandwidth limitations and difficulties in using short SIMD engines effectively. This work is about the techniques and a tool to improve in-core performance. Fundamental to the strategy is a hierarchical data layout made of small cubical structures of the problem states that can fit well in the cache hierarchy. The difficulties in computing the spatial derivatives (also called nearneighbor computation in the literature) in a hierarchical data layout are well known, hence, such a data layout has rarely been used in finite difference codes. This work discusses how to program relatively easily for such a hierarchical data layout, the inefficiencies in this programming strategy, and how to overcome the inefficiencies.The key technique to eliminate the overheads is called pipelinefor-reuse. It is followed by a storage optimization called maximal array contraction. Both pipeline-for-reuse and maximal array contraction are highly tedious and error-prone. Therefore, we built a source-to-source translator called CFD Builder to automate the transformations using directives. The directivebased approach leverages domain experts' knowledge about the code, and eliminates the need for complex analysis before program transformations. We demonstrated the effectiveness of this approach using three different applications on two different architectures and two different compilers. We see up to 6.92× performance improvement using such an approach. We believe such an approach could enable library and application writers to build efficient CFD libraries.

An extreme form of pipelining of the Piecewise-Parabolic Method (PPM) gas dynamics code has been used to dramatically increase its performance on the new generation of multicore CPUs. Exploiting this technique, together with a full integration of the several data post-processing and visualization utilities associated with this code has enabled numerical experiments in computational fluid dynamics to be performed interactively on a new, dedicated system in our lab, with immediate, user controlled visualization of the resulting flows on the PowerWall display. The code restructuring required to achieve the necessary CPU performance boost, as well as the parallel computing methods and systems used to enable interactive flow simulation are described. Requirements for these techniques to be applied to other codes are discussed, and our plans for tools that will assist programmers to exploit these techniques are briefly described. Examples showing the capability of the new system and software are given for applications in turbulence and stellar convection.