Granular material in a swirled container exhibits a curious transition as the number of particles is increased: at low densities the particle cluster rotates in the same direction as the swirling motion of the container, while at high densities it rotates in the opposite direction. We investigate this phenomenon experimentally and numerically using a co-rotating reference frame in which the system reaches a statistical steady-state. In this steady-state the particles form a cluster whose translational degrees of freedom are stationary, while the individual particles constantly circulate around the cluster's center of mass, similar to a ball rolling along the wall within a rotating drum. We show that the transition to counterrotation is friction-dependent. At high particle densities, frictional effects result in geometric frustration which prevents particles from cooperatively rolling and spinning. Consequently, the particle cluster rolls like a rigid body with no-slip conditions on the container wall, which necessarily counterrotates around its own axis. Numerical simulations verify that both wall-disc friction and disc-disc friction are critical for inducing counterrotation. arXiv:1804.02073v3 [cond-mat.soft]