Partial likelihood for signal processing

Abstract

We present partial likelihood (PL) as an effective means for developing nonlinear techniques for signal processing. Posing signal processing problems in a likelihood setting provides a number of advantages, such as allowing the use of powerful tools in statistics and easy incorporation of model order/complexity selection into the problem by use of appropriate information-theoretic criteria. However, likelihood formulations in most time series applications require a mechanism to discount the dependence structure of the data. We address how PL bypasses this requirement and note that it might coincide with conditional likelihood in a number of cases. We show that PL theory can also be used to establish the fundamental information-theoretic connection, to show the equivalence of likelihood maximization and relative entropy minimization without making the assumption of independent observations, which is an unrealistic assumption for most signal processing applications. We show that this equivalence is true for the basic class of probability models (the exponential family), which includes many important structures that can be used as nonlinear filters. We conclude by giving examples of the application of PL theory.

Automatic Bayesian Classification of Healthy Controls, Bipolar Disorder, and Schizophrenia Using Intrinsic Connectivity Maps From fMRI Data

“…This work is based on the CE SSB cost function, defined as follows: $$C_{\mathrm{CE}}(\mathrm{boldy},\mathrm{boldd})=-\sum _{i=1}^L{d}_i\text{log}{y}_i$$ obtained from (7) by using the definition of the Shannon entropy $h\left(\mathrm{boldy}\right)=-{\sum }_i{y}_i\text{log}{y}_i$ and the probability constraint ${\sum }_i{y}_i=1$, thus we can omit the term $-{\sum }_i(d_i-y_i)$. This choice is justified for two main reasons: first, the − CE has shown several advantages over other SSB cost functions, such us the square error (SE) cost function [32]–[34], and second, the minimization of the CE is equivalent to maximum likelihood (ML) estimation of the posterior class probabilities, [35]. Hence, large sample type arguments can be involved for steps such as order selection.…”

Asymptotic Analysis of Marginal-Likelihood Based Estimators for m-Dependent Processes

“…case. In [5], some of the statistical information on the temporal dependence is dropped in order to achieve an online causal estimation techniques. This misspecification is investigated using the partial likelihood theory [6].…”