Shannon entropy estimation for linear processes

Abstract

In this paper, we estimate the Shannon entropy S(f ) = −E [log(f (x))] of a one-sided linear process with probability density function f (x). We employ the integral estimator S n (f ), which utilizes the standard kernel density estimator f n (x) of f (x). We show that S n (f ) converges to S(f ) almost surely and in L 2 under reasonable conditions.