In this note, we consider the optimization problem associated with computing the rank decomposition of a symmetric tensor. We show that, in a well-defined sense, minima in this highly nonconvex optimization problem break the symmetry of the target tensor-but not too much. This phenomenon of symmetry breaking applies to various choices of tensor norms, and makes it possible to study the optimization landscape using a set of recently-developed symmetry-based analytical tools. The fact that the objective function under consideration is a multivariate polynomial allows us to apply symbolic methods from computational algebra to obtain more refined information on the symmetry breaking phenomenon. * This work was performed prior to joining Amazon. 1 Note that for odd-order tensors, problem (1) is equivalent to the standard tensor rank decomposition (also known as the real symmetric canonical polyadic decomposition (CPD)). Treatment of even-order tensors, where any linear combination of rank-1 tensors is allowed, is deferred to future work.2 For approaches for studing random nonconvex landscapes see, e.g., [8,9].