In Germany's D-Grid project numerous Grid communities are working together to providing a common overarching Grid infrastructure. The major aims of D-Grid are the integration of existing Grid deployments and their interoperability. The challenge in this endeavor lies in the heterogeneity of the current implementations: Three Grid middleware stacks and different Virtual Organization management approaches have to be embraced to achieve the intended goals. In this article we focus on the implementation of an attribute-based authorization infrastructure that not only leverages the well-known VO attributes but also Campus attributes managed by a Shibboleth federation.