A system with equation and dynamic boundary condition of Cahn-Hilliard type is considered. This system comes from a derivation performed in Liu-Wu (Arch. Ration. Mech. Anal., 233:167-247, 2019) via an energetic variational approach. Actually, the related problem can be seen as a transmission problem for the phase variable in the bulk and the corresponding variable on the boundary. The asymptotic behavior as the coefficient of the surface diffusion acting on the boundary phase variable goes to 0 is investigated. By this analysis we obtain a forward-backward dynamic boundary condition at the limit. We can deal with a general class of potentials having a double-well structure, including the non-smooth double-obstacle potential. We illustrate that the limit problem is well-posed by also proving a continuous dependence estimate. Moreover, in the case when the two graphs, in the bulk and on the boundary, exhibit the same growth, we show that the solution of the limit problem is more regular and we prove an error estimate for a suitable order of the diffusion parameter.