Low complexity sequential lossless coding for piecewise stationary memoryless sources

Shamir, G.I.; Merhav, N.

doi:10.1109/isit.1998.708627

Cited by 15 publications

(32 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though the results in Merhav [45], Willems [46], and Shamir and Merhav [47] are for p.i.i.d. sources, it is easy to check that if (1) holds, then all of their results go through.…”

Section: E Coding For Piecewise-constant Parametersmentioning

confidence: 87%

See 1 more Smart Citation

Universal lossless source coding with the Burrows Wheeler transform

Effros

Visweswariah

Kulkarni

et al. 2002

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-The Burrows Wheeler Transform (BWT) is a reversible sequence transformation used in a variety of practical lossless source-coding algorithms. In each, the BWT is followed by a lossless source code that attempts to exploit the natural ordering of the BWT coefficients. BWT-based compression schemes are widely touted as low-complexity algorithms giving lossless coding rates better than those of the Ziv-Lempel codes (commonly known as LZ'77 and LZ'78) and almost as good as those achieved by prediction by partial matching (PPM) algorithms. To date, the coding performance claims have been made primarily on the basis of experimental results. This work gives a theoretical evaluation of BWT-based coding. The main results of this theoretical evaluation include: 1) statistical characterizations of the BWT output on both finite strings and sequences of length , 2) a variety of very simple new techniques for BWT-based lossless source coding, and 3) proofs of the universality and bounds on the rates of convergence of both new and existing BWT-based codes for finite-memory and stationary ergodic sources. The end result is a theoretical justification and validation of the experimentally derived conclusions: BWT-based lossless source codes achieve universal lossless coding performance that converges to the optimal coding performance more quickly than the rate of convergence observed in Ziv-Lempel style codes and, for some BWT-based codes, within a constant factor of the optimal rate of convergence for finite-memory sources.Index Terms-Burrows Wheeler Transform (BWT), rate of convergence, redundancy, text compression, universal noiseless source coding.

show abstract

“…Even though the results in Merhav [45], Willems [46], and Shamir and Merhav [47] are for p.i.i.d. sources, it is easy to check that if (1) holds, then all of their results go through.…”

Section: E Coding For Piecewise-constant Parametersmentioning

confidence: 87%

“…The space complexities of the two algorithms grow more slowly than the time complexities, which are and , respectively. In [47], Shamir and Merhav describe an algorithm giving…”

Section: E Coding For Piecewise-constant Parametersmentioning

confidence: 99%

Universal lossless source coding with the Burrows Wheeler transform

Effros

Visweswariah

Kulkarni

et al. 2002

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…The idea of slowly decreasing the switching rate was considered in [12] in the context of source coding, and later analysed for expert switching in [10]; we saw in Section 3.2 that it also underlies Follow the Leading History of [7]. It results in tracking regret bounds that are almost as good as the bounds for constant α with optimally tuned α.…”

Section: Theorem 4 the Worst-case Adaptive Regret Of Fixed Share Withmentioning

confidence: 99%

A Closer Look at Adaptive Regret

Adamskiy

Koolen

Chernov

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. For the prediction with expert advice setting, we consider methods to construct algorithms that have low adaptive regret. The adaptive regret of an algorithm on a time interval [t1, t2] is the loss of the algorithm there minus the loss of the best expert. Adaptive regret measures how well the algorithm approximates the best expert locally, and it is therefore somewhere between the classical regret (measured on all outcomes) and the tracking regret, where the algorithm is compared to a good sequence of experts. We investigate two existing intuitive methods to derive algorithms with low adaptive regret, one based on specialist experts and the other based on restarts. Quite surprisingly, we show that both methods lead to the same algorithm, namely Fixed Share, which is known for its tracking regret. Our main result is a thorough analysis of the adaptive regret of Fixed Share. We obtain the exact worst-case adaptive regret for Fixed Share, from which the classical tracking bounds can be derived. We also prove that Fixed Share is optimal, in the sense that no algorithm can have a better adaptive regret bound.

show abstract

Section: Introductionmentioning

confidence: 99%

“…In this paper, we show that the performance weighting approach used in [1], [5] can be generalized to yield algorithms that can efficiently compete with the best partition over all partitions for more general loss functions. We demonstrate that the universal probability assignment introduced in [1], [5] can be effectively merged into the prediction framework by using the methodology introduced in [2]. Although we investigate the continuous class of linear regressors or that of a finite number of adaptive filters as our competition class and use the square error loss, the methodology introduced in this paper can be extended to arbitrary competition classes, such as that of certain nonlinear regressors considered in [6], [7] or to more general loss functions as in [8].…”

Section: Introductionmentioning

confidence: 99%

Universal Switching Linear Least Squares Prediction

Kozat

Singer

2008

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Abstract-We consider sequential regression of individual sequences under the square error loss. Using a competitive algorithm framework, we construct a sequential algorithm that can achieve the performance of the best piecewise (in time) linear regression algorithm tuned to the underlying individual sequence. The sequential algorithm we construct does not need the data length, number of piecewise linear regions, or the locations of the transition times, however, it can asymptotically achieve the performance of the best piecewise (in time) linear regressor that can choose number of segments, duration of these segments and best regressor in each segment, based on observation of the whole sequence in advance. We use a transition diagram similar to that of [Willems '96] to effectively combine an exponential number of competing algorithms, with a complexity that is only linear in the data length. We demonstrate that the regret of this approach is at most O(4 ln(n)) per transition for not knowing the best transition times and at most O(ln(n)) for estimating the best linear regressor in each segment, where n is the total length of the observation process. Lower bounds for any sequential algorithm demonstrate a form of minmax optimality in certain settings. We then extend these results to include a finite collection of competing algorithms within each time segment, rather than linear regressors. I. INTRODUCTIONA common approach to applications in adaptive signal processing is to take the viewpoint of turning the problem at hand, such as equalization, prediction or some other sequential decision problem, and turn it into an associated parametric modeling or estimation problem. By forcing the problem into this form, we then have to live with the associated performance of the resulting parameter estimation problem, which in general is worse, often significantly worse, than that which could have been obtained by directly addressing the problem at hand. Moreover, if the assumptions in the model do not match reality, then the performance of the algorithm tuned to the assumed statistical model may deteriorate considerably.In this paper, we approach the problem of prediction from a competitive algorithm point of view. By defining a competitive framework, we try to achieve the performance of the best algorithm from a large class of candidate algorithms, rather than attempting to fit a given model to the data at hand. The performance measure of interest is then defined with respect to the best from this class, instead of the usual parametric modeling error between the output of the modeling algorithm and the desired signal directly. We will show that by not forcing the algorithms to make hard decisions about a set of parameters at each step, but rather permitting a competition among many candidate models, we can obtain algorithms that

show abstract

Low complexity sequential lossless coding for piecewise stationary memoryless sources

Cited by 15 publications

References 15 publications

Universal lossless source coding with the Burrows Wheeler transform

Universal lossless source coding with the Burrows Wheeler transform

A Closer Look at Adaptive Regret

Universal Switching Linear Least Squares Prediction

Contact Info

Product

Resources

About