Subband coding for large-scale scientific simulation data using JPEG 2000

Abstract

Abstract-The ISO/IEC JPEG 2000 image coding standard is a family of source coding algorithms targeting high-resolution image communications. JPEG 2000 features highly scalable embedded coding features that allow one to interactively zoom out to reduced resolution thumbnails of enormous data sets or to zoom in on highly localized regions of interest with very economical communications and rendering requirements. While intended for fixed-precision input data, the implementation of the irreversible version of the standard is often done internally in floating point arithmetic. Moreover, the standard is designed to support high-bit-depth data. Part 2 of the standard also provides support for three-dimensional data sets such as multicomponent or volumetric imagery. These features make JPEG 2000 an appealing candidate for highly scalable communications coding and visualization of two-and three-dimensional data produced by scientific simulation software. We present results of initial experiments applying JPEG 2000 to scientific simulation data produced by the Parallel Ocean Program (POP) global ocean circulation model, highlighting both the promise and the many challenges this approach holds for scientific visualization applications.

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

“…In situ analysis algorithms may transform data into reduced representations or surrogate models in order to mitigate large data size, high dimensionality, or long computation times. Low-rank approximation (Austin et al, 2016), statistical summarization (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Lawrence et al, 2017;Lohrmann et al, 2017;Thompson et al, 2011), topological segmentation (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelet transformation (Li et al, 2017;Salloum et al, 2018), lossy compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), geometric modeling (Nashed et al, 2019; Peterka et al, 2018), and feature detection (Guo et al, 2017) may be used to generate reduced or surrogate models.…”

“…Research is required to modify existing post hoc algorithms and develop new in situ algorithms to satisfy the needs of modern use cases on emerging system architectures that can feature massive scale, many cores, deep memory hierarchies, or embedded lightweight edge devices. Examples of such algorithms include reduced representations and low-rank approximations (Austin et al, 2016), statistical (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Thompson et al, 2011), topological (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelets (Li et al, 2017;Salloum et al, 2018), compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), and feature detection (Guo et al, 2017) methods. Surrogate models and multifidelity models can be geometric (Nashed et al, 2019;Peterka et al, 2018), statistical (Lawrence et al, 2017;Lohrmann et al, 2017), or neural network (He et al, 2019).…”