Abstract. We present near-IR (J and Ks) number counts and colors of galaxies detected in deep VLT-ISAAC images centered on the Chandra Deep Field and Hubble Deep Field-South for a total area of 13.6 arcmin 2 . The limiting surface brightness obtained is Ks 22.8 mag/arcsec 2 and J 24.5 (1σ) on both fields. A d log N/dm relation with a slope of ∼0.34 in J and ∼0.28 in Ks is found in both fields with no evidence of decline near the magnitude limit. The median J − Ks color of galaxies becomes bluer at magnitudes fainter than Ks ∼ 18, in agreement with the different number counts slope observed in the two bands. We find a fraction (≤5% of the total sample) of sources with color redder than J − Ks = 2.3 at magnitudes Ks > 20. Most of them appear as isolated sources, possibly elliptical or dusty starburst galaxies at redshift z > 2. The comparison of the observed number counts with models shows that our J-band and Ks-band counts are consistent with the prediction of a model based on a small amount of merging in a Ω = 1 cosmology. On the other hand, we fail to reproduce the observed counts if we do not consider merging independently of the parameters defining the universe.