Convexity and star-shapedness of real linear images of special orthogonal orbits

Abstract

Let A ∈ R n×n and SO n := {U ∈ R n×n : UU t = I n , detU > 0} be the set of n × n special orthogonal matrices. Define the (real) special orthogonal orbit of A byIn this paper, we show that the linear image of O(A) is starshaped with respect to the origin for arbitrary linear maps L : R n×n → R if n ≥ 2 −1 . In particular, for linear maps L : R n×n → R 2 and when A has distinct singular values, we study B ∈ O(A) such that L(B) is a boundary point of L (O(A)). This gives an alternative proof of a result by Li and Tam on the convexity of L(O(A)) for linear maps L : R n×n → R 2 .