A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time

Abstract

This article presents an overview of a theory for performing temporal smoothing on temporal signals in such a way that: (i) temporally smoothed signals at coarser temporal scales are guaranteed to constitute simplifications of corresponding temporally smoothed signals at any finer temporal scale (including the original signal) and (ii) the temporal smoothing process is both time-causal and time-recursive, in the sense that it does not require access to future information and can be performed with no other temporal memory buffer of the past than the resulting smoothed temporal scale-space representations themselves. For specific subsets of parameter settings for the classes of linear and shift-invariant temporal smoothing operators that obey this property, it is shown how temporal scale covariance can be additionally obtained, guaranteeing that if the temporal input signal is rescaled by a uniform temporal scaling factor, then also the resulting temporal scale-space representations of the rescaled temporal signal will constitute mere rescalings of the temporal scale-space representations of the original input signal, complemented by a shift along the temporal scale dimension. The resulting time-causal limit kernel that obeys this property constitutes a canonical temporal kernel for processing temporal signals in real-time scenarios when the regular Gaussian kernel cannot be used, because of its non-causal access to information from the future, and we cannot additionally require the temporal smoothing process to comprise a complementary memory of the past beyond the information contained in the temporal smoothing process itself, which in this way also serves as a multi-scale temporal memory of the past. We describe how the time-causal limit kernel relates to previously used temporal models, such as Koenderink’s scale-time kernels and the ex-Gaussian kernel. We do also give an overview of how the time-causal limit kernel can be used for modelling the temporal processing in models for spatio-temporal and spectro-temporal receptive fields, and how it more generally has a high potential for modelling neural temporal response functions in a purely time-causal and time-recursive way, that can also handle phenomena at multiple temporal scales in a theoretically well-founded manner. We detail how this theory can be efficiently implemented for discrete data, in terms of a set of recursive filters coupled in cascade. Hence, the theory is generally applicable for both: (i) modelling continuous temporal phenomena over multiple temporal scales and (ii) digital processing of measured temporal signals in real time. We conclude by stating implications of the theory for modelling temporal phenomena in biological, perceptual, neural and memory processes by mathematical models, as well as implications regarding the philosophy of time and perceptual agents. Specifically, we propose that for A-type theories of time, as well as for perceptual agents, the notion of a non-infinitesimal inner temporal scale of the temporal receptive fields has to be included in representations of the present, where the inherent nonzero temporal delay of such time-causal receptive fields implies a need for incorporating predictions from the actual time-delayed present in the layers of a perceptual hierarchy, to make it possible for a representation of the perceptual present to constitute a representation of the environment with timing properties closer to the actual present.

Covariance properties under natural image transformations for the generalised Gaussian derivative model for visual receptive fields

“…These semi-group properties do, in turn, mean that spatial smoothed representations with the rotationally symmetric Gaussian kernel are related according to the spatial cascade smoothing property L(x; s 2 ) = g(x; s 2 − s 1 ) * L(x; s 1 ), provided that s 2 > s 1 , whereas spatially smoothed representations with the a ne Gaussian kernel are related according to spatial cascade smoothing property L(x; 2 ) = g(x; 2 − 1 ) * L(x; 1 ), coarser scale representations from finer scale levels by adding complementary spatial smoothing stages in cascade. The timecausal limit kernel ( 14) used for temporal smoothing in our spatiotemporal receptive field model does also obey a cascade smoothing property over temporal scales, which makes it possible to compute representations at coarser temporal scales from representations at finer spatial scales, by complementary (time-causal) temporal filtering, in terms of first-order temporal integrators coupled in cascade (Lindeberg, 2023b). Do the earliest layers in a biological visual system explicitly represent the image and video data by expansions of receptive fields over multiple spatial and temporal scales, or do the earliest stages in the vision system instead only represent a lowest range of spatial and temporal scales explicitly, to then handle coarser spatial and temporal scales by other mechanisms?…”

Artificial Intelligence-Based Algorithms in Medical Image Scan Segmentation and Intelligent Visual Content Generation—A Concise Overview

“…This mechanism causes ions to diffuse in the direction of lowering the potential to the resting level or another level U 0 → U leak < U thr . Thus, the third generation of neural networks, i.e., the spiking neural networks (SNNs) [38], are mostly based on the LIF, where the membrane potential U(t) is determined by the equation…”

Effects of RF Signal Eventization Encoding on Device Classification Performance

“…The reader is referred to the many works that have addressed GTX development and its application for improving signal joint time-frequency resolution. The GTX application trail is diverse and spans from some of the earliest signal processing development activity [37][38][39] to some o the most recent work [40,41]. Of most relevance here is the Gabor-based DNA fingerprint ing work, with early demonstrations occurring in [42,43] and the most recent demonstra tions being performed in [8]-the GTX processing implemented here was based on these works.…”