We study the Cramér type moderate deviation for partial sums of random fields by applying the conjugate method. The results are applicable to the partial sums of linear random fields with short or long memory and to nonparametric regression with random field errors.

In this paper, it is shown that $$C_\beta $$ C β -smooth functions can be approximated by deep neural networks with ReLU activation function and with parameters $$\{0,\pm \frac{1}{2}, \pm 1, 2\}$$ { 0 , ± 1 2 , ± 1 , 2 } . The $$l_0$$ l 0 and $$l_1$$ l 1 parameter norms of considered networks are thus equivalent. The depth, the width and the number of active parameters of the constructed networks have, up to a logarithmic factor, the same dependence on the approximation error as the networks with parameters in $$[-1,1]$$ [ - 1 , 1 ] . In particular, this implies that the nonparametric regression estimation with constructed networks achieves, up to logarithmic factors, the same minimax convergence rates as with sparse networks with parameters in $$[-1,1]$$ [ - 1 , 1 ] .

An example of an activation function σ is given such that networks with activations {σ, ⌊•⌋}, integer weights and a fixed architecture depending on d approximate continuous functions on [0, 1] d . The range of integer weights required for ε-approximation of Hölder continuous functions is derived, which leads to a convergence rate of order n −2β 2β+d log 2 n for neural network regression estimation of unknown β-Hölder continuous function with given n samples.

We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The estimates enjoy a central limit theorem under some regular conditions on the distributions. The theoretical results are demonstrated by simulation study.