Compressing three-dimensional GRIB meteorological data using KLT and JPEG 2000

Lucero, A.; Cabrera, Sergio D.; Aguirre, Antônio; Vidal, E.

doi:10.1109/igarss.2003.1294266

Cited by 8 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While lossless or near-lossless compression has been a major focus due to the need to preserve all fidelity, it is worthwhile to explore lossy compression techniques to evaluate the potential rate/distortion performance on ultraspectral data. Similar work has been previously done by our group for other kinds of data such as hyper-spectral imagery, modeled meteorological data and medical multi-slice scans [1][2][3][6][7][8][9]. In a similar manner, this study presents an assessment of inter-band bit-allocation strategies applied to AIRS data using a JPEG2000 compatible compression scheme for multicomponent imagery.…”

Section: Introductionmentioning

confidence: 62%

Assessment of inter-slice bit-allocation strategies in the application of JPEG2000 to ultraspectral sounder data

2006

View full text Add to dashboard Cite

Presented here is a study comparing various inter-slice bit-allocation strategies when applying JPEG2000 part 2 to ultraspectral sounder satellite data. Two families of algorithms are compared: the first is strictly variance based for which traditional approaches and some variations are used; the second uses rate distortion curves (RDCs) to optimally allocate slice bit-rates in the MSE sense, analogous to the way the PCRD algorithm uses RDCs of code-blocks in the baseline JPEG2000 method. The rate distortion curves are either experimentally gathered or analytically modeled using techniques discussed in previous papers. An additional approach to gather RDCs is considered using the slope-length information computed by a JPEG2000 encoder. The study is done using six different granules of ultraspectral sounder data made available by NOAA. Every subset contains 1,501 spectral bands, each consisting of 90 by 135 pixels, which are decorrelated using either a Karhunen-Loève transform (KLT) or the discrete wavelet transform (DWT). The most complex RDC-based optimal approach is used to establish a lower distortion bound, in addition to characterizing the typical distortion performance achievable when compressing spectrally decorrelated sounder data with JPEG2000. The bound is used as the basis of comparison against the other methods studied. Moreover, the software tool CompressMD developed by our group is used to provide all data handling, algorithm implementations and collection of results.

show abstract

Section: Introductionmentioning

confidence: 62%

Assessment of inter-slice bit-allocation strategies in the application of JPEG2000 to ultraspectral sounder data

2006

View full text Add to dashboard Cite

show abstract

“…A note on the Hyperion data is that the black strips that can be observed in Figure 2b are due to missing information (zero or constant data) in those corresponding bands. A similar analysis on meteorological data can be observed in [15]. Another way of analyzing the benefit of using a transform, in this case the KLT, in the z-direction is to look at the distribution of the variance in the transform domain.…”

Section: Discussionmentioning

confidence: 87%

“…The first 5 slices of the Cuprite and Hyperion data are shown in Figures 4a and 4b, respectively, to provide a view of what the data being compressed looks like when considered as images. Another measure, the coding gain, could be used (not shown here, see [15]) in order to see if applying a transform in the z-direction would help improve the compression results. The procedure used to compress the data is described in the following section.…”

Section: Discussionmentioning

confidence: 99%

<title>Evaluating residual coding with JPEG2000 for L-infinity driven hyperspectral image compression</title>

et al. 2005

View full text Add to dashboard Cite

This paper presents a study on the compression of hyperspectral satellite data using JPEG 2000 and residual encoding (RE). The first step in the process is to apply a decorrelating transform in the spectral-direction or z-direction. In most cases in this study, the Karhunen -Loeve Transform (KLT) is used. For comparison, some examples are also included where the discrete wavelet transform (DWT) is used for this purpose as well as examples with a purely 2-D approach that uses no z-direction transform. Bit-rate allocation techniques are used in order to take advantage of the energy compaction obtained when applying a transform in the z-direction. The transformed slices and their corresponding bit rates are input into JPEG 2000 in order to obtain the compressed bit stream. In this study, the compressed bit stream is decompressed at the encoder side in order to compute the recovered data. These data are then subtracted from the original data in order to calculate the residuals, which are then quantized and losslessly encoded separately using JPEG2000 itself in order to control the maximum absolute error (MAE). An analysis between using and omitting residual encoding with respect to MAE is included. It is observed that a decrease in the MAE by a factor of 3 is achieved for this data with very small overhead when the residual encoding is utilized. The two data sets used in this study are the well known Cuprite radiance imagery from AVIRIS and a set from the Hyperion satellite system, both of which are available in 16 bits per value form.

show abstract

“…Version 1 and earlier versions of CompressMD have been used to do much of our previous experimentation in [5], [6], [7], [8], [9]. This software uses several open source packages, JFreeChart provides XY plotting facilities, Colt & JMAT provide matrix computations, Netcdf & Grib readers and writers provide for file I/O.…”

Section: Java Software Toolsmentioning

confidence: 99%

<title>JAVA implemented MSE optimal bit-rate allocation applied to 3-D hyperspectral imagery using JPEG2000 compression</title>

et al. 2005

Self Cite

View full text Add to dashboard Cite

This paper describes an efficient algorithm and its Java implementation for a recently developed mean-squared error (MSE) rate-distortion optimal (RDO) inter-slice bit-rate allocation (BRA) scheme applicable to the JPEG2000 Part 2 (J2KP2) framework. Its performance is illustrated on hyperspectral imagery data using the J2KP2 with the KarhunenLoeve transform (KLT) for decorrelation. The results are contrasted with those obtained using the traditional logvariance based BRA method and with the original RDO algorithm. The implementation has been developed as a Java plug-in to be incorporated into our evolving multi-dimensional data compression software tool denoted CompressMD. The RDO approach to BRA uses discrete rate distortion curves (RDCs) for each slice of transform coefficients. The generation of each point on a RDC requires a full decompression of that slice, therefore, the efficient version minimizes the number of RDC points needed from each slice by using a localized coarse-to-fine approach denoted RDOEfficient. The scheme is illustrated in detail using a subset of 10 bands of hyperspectral imagery data and is contrasted to the original RDO implementation and the traditional (log-variance) method of BRA showing that better results are obtained with the RDO methods. The three schemes are also tested on two hyperspectral imagery data sets with all bands present: the Cuprite radiance data from AVIRIS and a set derived from the Hyperion satellite. The results from the RDO and RDOEfficient are very close to each other in the MSE sense indicating that the adaptive approach can find almost the same BRA solution. Surprisingly, the traditional method also performs very close to the RDO methods, indicating that it is very close to being optimal for these types of data sets.

show abstract

Compressing three-dimensional GRIB meteorological data using KLT and JPEG 2000

Cited by 8 publications

References 5 publications

Assessment of inter-slice bit-allocation strategies in the application of JPEG2000 to ultraspectral sounder data

Assessment of inter-slice bit-allocation strategies in the application of JPEG2000 to ultraspectral sounder data

<title>Evaluating residual coding with JPEG2000 for L-infinity driven hyperspectral image compression</title>

<title>JAVA implemented MSE optimal bit-rate allocation applied to 3-D hyperspectral imagery using JPEG2000 compression</title>

Contact Info

Product

Resources

About