We study the system of heterogeneous interbank lending and borrowing based on the relative average of log-capitalization given by the linear combination of the average within groups and the ensemble average and describe the evolution of log-capitalization by a system of coupled diffusions. The model incorporates a game feature with homogeneity within groups and heterogeneity between groups where banks search for the optimal lending or borrowing strategies through minimizing the heterogeneous linear quadratic costs in order to avoid to approach the default barrier. Due to the complicity of the lending and borrowing system, the closed-loop Nash equilibria and the open-loop Nash equilibria are both driven by the coupled Riccati equations. The existence of the equilibria in the two-group case where the number of banks are sufficiently large is guaranteed by the solvability for the coupled Riccati equations as the number of banks goes to infinity in each group. The equilibria are consisted of the mean-reverting term identical to the one group game and the group average owing to heterogeneity. In addition, the corresponding heterogeneous mean filed game with the arbitrary number of groups is also discussed. The existence of the ǫ-Nash equilibrium in the general d heterogeneous groups is also verified. Finally, in the financial implication, we observe the Nash equilibria governed by the mean-reverting term and the linear combination of the ensemble averages of individual groups and study the influence of the relative parameters on the liquidity rate through the numerical analysis.