Abstract-Noise can improve how memoryless neurons process signals and maximize their throughput information. Such favorable use of noise is the so-called "stochastic resonance" or SR effect at the level of threshold neurons and continuous neurons. This paper presents theoretical and simulation evidence that 1) lone noisy threshold and continuous neurons exhibit the SR effect in terms of the mutual information between random input and output sequences, 2) a new statistically robust learning law can find this entropy-optimal noise level, and 3) the adaptive SR effect is robust against highly impulsive noise with infinite variance. Histograms estimate the relevant probability density functions at each learning iteration. A theorem shows that almost all noise probability density functions produce some SR effect in threshold neurons even if the noise is impulsive and has infinite variance. The optimal noise level in threshold neurons also behaves nonlinearly as the input signal amplitude increases. Simulations further show that the SR effect persists for several sigmoidal neurons and for Gaussian radial-basis-function neurons.Index Terms-Alpha-stable noise, impulsive noise, infinite-variance statistics, mutual information, noise processing, sigmoidal neurons and radial basis functions, stochastic gradient learning, stochastic resonance (SR), threshold neurons.
I. NOISE AND ADAPTIVE STOCHASTIC RESONANCE
NOISE is an unwanted signal or source of energy. Scientists and engineers have largely tried to filter noise or cancel it or mask it out of existence. The Noise Pollution Clearinghouse condemns noise outright: "Noise is unwanted sound. It is derived from the Latin word 'nausea' meaning seasickness. Noise is among the most pervasive pollutants today. Noise from road traffic, jet planes, jet skis, garbage trucks, construction equipment, manufacturing processes, lawn mowers, leaf blowers, and boom boxes, to name a few, are among the unwanted sounds that are routinely broadcast into the air."The Fig. 1 shows how uniform pixel noise can improve our subjective perception of an image. The system quantizes the original gray-scale "Lena" image into a binary image of black and white pixels. It emits a white pixel as output if the input gray-scale pixel equals or exceeds a threshold. It emits a black pixel as output if the input gray-scale pixel falls below the threshold. This quantizer is biased because it does not set the threshold at the midpoint of the gray scale. So the quantized version of the original image contains almost no information. A small level of noise sharpens the image contours and helps fill in features when it adds to the original image before the system applies the threshold. Too much noise swamps the image and degrades its contours. Gammaitoni [29] and others [70] have proposed a dithering argument for this SR effect and still others [55] have applied this argument to still images. The argument involves adding dither noise to a signal before quantization. Consider gray-scale pixel and binary output pixel wi...