In this paper, we investigate a variational polycrystalline model in finite crystal plasticity with one active slip system and rigid elasticity. The task is to determine inner and outer bounds on the domain of the constrained macroscopic elastoplastic energy density, or equivalently, the affine boundary values of a related inhomogeneous differential inclusion problem. A geometry-independent Taylor inner bound, which we calculate directly, follows from considering constant-strain solutions to a relaxed problem in combination with well-known relaxation and convex integration results. On the other hand, we deduce outer bounds from a rank-one compatibility condition between the affine boundary data and the microscopic strain at the boundary grains. While there are examples of polycrystals for which the two above-mentioned bounds coincide, we present an explicit construction to prove that the Taylor bound is non-optimal in general.