Graphdiyne (GDY), with uniform pores and atomic thickness, is attracting widespread attention for application in H2 separation in recent years. However, the challenge lies in the rational design of GDYs for fast and selective H2 permeation. By MD and DFT calculations, several flexible GDYs were constructed to investigate the permeation properties of four pure gas (H2, N2, CO2, and CH4) and three equimolar binary mixtures (H2/N2, H2/CO2, and H2/CH4) in this study. When the pore size is smaller than 2.1 Å, the GDYs acted as an exceptional filter for H2 with an approximately infinite H2 selectivity. Beyond the size-sieving effect, in the separation process of binary mixtures, the blocking effect arising from the strong gas–membrane interaction was proven to greatly impede H2 permeation. After understanding the mechanism, the H2 permeance of the mixtures of H2/CO2 was further increased to 2.84 × 105 GPU by reducing the blocking effect with the addition of a tiny amount of surface charges, without sacrificing the selectivity. This theoretical study provides an additional atomic understanding of H2 permeation crossing GDYs, indicating that the GDY membrane could be a potential candidate for H2 purification.