Maximization of ESI. Jaynes principle in testing significant inputs of linear model

Abstract

Search for an unknown S-tuple A of significant inputs of a linear model with random IID •discrete binary carriers and finitely supported IID noise is studied. Two statistics T\ , T" s , based on maximization of Shannon Information of the joint empirical input-output distributions, are proposed inspired by the related study in Csiszar and K rner (1981). The first one compares N-sequences of teach input and of the output separately. The second one explores the relation between 5-tuples of J jV-columns of the (N X t) design matrix and the output sequence. N is the number of experiments land t is the total number of inputs. Both statistics are shown to be asymptotically as efficient as 'the ML-test based on the same two classes of joint empirical distributions in the artificial case when ML-test is applicable: if the unknown parameters 6λ, λ G A, of the model and the distribution of -errors are known. Our tests do not require this information appearing asymptotically uniformly most I efficient in the corresponding classes of tests. T s is shown to provide asymptotically the best rate of search for the set A but requires about t s log t cycles of computing. This may appear inaccessible : in some applications. T\ requires only t log t cycles of computing operations and has the rate of T the best order of magnitude. These results essentially resolve the problem of asymptotically efficient analysis of scatter diagrams posed in Random Balance Method (Budne (1959)). Results of simulation -are presented confirming theoretical results.with random jointly IID carriers Xi(X) € Β = {-1,1}, ζ* (λ) = 1 with probability /?, Ο < β < 1, λ = l, . . . , t] i = l, ... , N and IID random errors e*, i = l, . . . , N. Assume that b\ = 0 unless λ G A with cardinality \A\ = s « t. RBM was proposed Brought to you by | University of Arizona Authenticated Download Date | 6/9/15 4:52 PM M. Malyutov and H. Sadakato detect the set A ( T. A. Budne (1959)) and applied successfully to numerous cases of finding disorders of industrial production.In the discussion contained in the same issue of the journal, RBM was unanimously rejected by the leading applied statisticians of that time. They seemed to overlook the difference of the set-up studied from the estimation of parameters in (1) requiring a new type of design and analysis.Actually, the visual inspection of scatter diagrams of data TV-sequences (X N (X), Z N ) for each = 1, ...,£, was proposed by the authors of RBM, although some rank statistics (e.g. Budne test) were also applied.The main point in their arguments was probably Fisher's idea of randomization: when the value of a variable is fixed, other variables' contribution to the output is a pure noise under the random design. Hence the influence of each variable on the output can be estimated in this noisy background. It was proved in Malyutov (1979Malyutov ( , 1983 for general finitely-valued models that the asymptotic rate of ML-test is maximal for certain random designs (see below).The relevant Bahadur efficiency of linear and other rank ...

Near-Optimal Noisy Group Testing via Separate Decoding of Items

“…In fact, the same turns out to be true for the symmetric noise model, matching (11) up to a factor of log 2, or even better if ν is further optimized (see Appendix A). In more recent works [17], [23], similar results were shown when the rule (9) is replaced by a universal rule (i.e., not depending on the noise distribution) based on the empirical mutual information. However, we stick to (9) in this paper, as we found it to be more amenable to general scalings of the form k = o(p).…”

“…The rule (9) requires knowledge of the Bernoulli test parameter ν, the crossover probability ρ, and the number of defectives k. The last of these poses the strongest assumption, though it is commonly made in the group testing literature. We leave the study of universal variants (e.g., see [23]) for future work. Figure 1 summarizes the main results known for the noiseless and symmetric noise models (both information-theoretic and practical), along with our novel contributions.…”

Group Testing: An Information Theory Perspective

“…In a follow-up work [142], similar results were shown when the rule (3.25) is replaced by a universal rule (one that does not depend on the noise distribution) based on the empirical mutual information.…”

“…In the sparse regime with α → 0, they show that their decoders achieve the same asymptotic performance as when the channel is known. An earlier work of Malyutov and Sadaka [142] showed such a result in the very sparse regime k = O(1).…”

“…Here a transmitter seeks to send a message over a noisy channel, despite not having precise channel statistics other than a guarantee that its capacity exceeds the transmission rate. The maximum empirical mutual information decoder (e.g., see [50, p. 100]) is the most well-known decoding method for this scenario, and such a decoder was in fact adopted for group testing in [142]. The decoder adopted in [105] is slightly different, but still based on empirical probability measures.…”