Grid systems provide a standard platform to share resources, such as computing facilities and backup services, between research communities and commercial organizations. Nowadays there are plenty of grids among different institutions, and many of them are probably heterogeneous to others because they are constructed by different types of middleware. Enabling all these grids to work as a single grid can provide users with even more computing power and services. Because of varieties of middleware and certification authority, cooperating across different grids is difficult. We will present a single sign-on scheme such that heterogeneous grids can cooperate through it on top of existing middleware. The proposed scheme focuses on achieving interoperability among different grids above a P2P layer. By means of the scheme and MyProxy X.509 credentials services, we demonstrate our solution to execute jobs on heterogeneous grids with single sign-on.