We present an approach for the inclusion of nonspherical constituents in high-resolution N-body discrete element method (DEM) simulations. We use aggregates composed of bonded spheres to model nonspherical components. Though the method may be applied more generally, we detail our implementation in the existing N-body code pkdgrav. It has long been acknowledged that nonspherical grains confer additional shear strength and resistance to flow when compared with spheres. As a result, we expect that rubble-pile asteroids will also exhibit these properties and may behave differently than comparable rubble piles composed of idealized spheres. Since spherical particles avoid some significant technical challenges, most DEM gravity codes have used only spherical particles or have been confined to relatively low resolutions. We also discuss the work that has gone into improving performance with nonspherical grains, building on pkdgrav's existing leading-edge computational efficiency among DEM gravity codes. This allows for the addition of nonspherical shapes while maintaining the efficiencies afforded by pkdgrav's tree implementation and parallelization. As a test, we simulated the gravitational collapse of 25,000 nonspherical bodies in parallel. In this case, the efficiency improvements allowed for an increase in speed by nearly a factor of 3 when compared with the naive implementation. Without these enhancements, large runs with nonspherical components would remain prohibitively expensive. Finally, we present the results of several small-scale tests: spin-up due to the YORP effect, tidal encounters, and the Brazil nut effect. In all cases, we find that the inclusion of nonspherical constituents has a measurable impact on simulation outcomes.