Reflection groups and a convolution theorem

Abstract

The purpose of this paper is to prove a converse to a theorem of Eaton and Perlman on convolutions of G-decreasing functions. Both their result and our converse concern a connection between the theory of reflection groups and a class of probability inequalities that are of interest to statisticians. The original theorem states that the Convolution Theorem is satisfied by reflection subgroups of the orthogonal group. We show in this paper that if G is a finite linear group that satisfies the Convolution Theorem, then G is a reflection group. Furthermore, we show that if ρ : G → GL(V ) is a faithful representation that satisfies the Convolution Theorem, then ρ is a direct sum of the canonical Coxeter representation of G and a trivial representation. Introduction.Let V be a finite dimensional real vector space, and let G → GL(V ) be a linear group. A function f : V → R is said be G-decreasing provided that f (x) ≥ f (y) whenever x is contained in the convex hull of the G-orbit of y. In [EP1], Eaton and Perlman showed that if V is a Euclidean space and G is the closure of a reflection subgroup (i.e., a subgroup generated by reflections) of the orthogonal group O(V ), then the convolution of a pair of G-decreasing real valued functions defined on V is itself a G-decreasing function. For this reason such groups are said to satisfy the Convolution Theorem.Complementing their convolution theorem, Eaton and Perlman showed inis the closure of a reflection subgroup of O(V ), then G can be decomposed into a direct product of a finite number of compact orthogonal groups and a finite reflection group; this decomposition also follows from the result on irreducible reflection groups cited above and the complete reducibility of representations of compact groups. Since the factors in the decomposition are reflection groups, the convolution theorem in [EP1] can be paraphrased as follows: The Convolution Theorem is satisfied by closed reflection subgroups of the orthogonal group O(V ). 229