The Kepler mission has yielded the discovery of eight circumbinary systems, all found around eclipsing binaries with periods greater than 7 d. This is longer than the typical eclipsing binary period found by Kepler, and hence there is a dearth of planets around the closest binaries. In this paper we suggest how this dearth may be explained by the presence of a distant stellar tertiary companion, which shrunk the inner binary orbit by the process of Kozai cycles and tidal friction, a mechanism that has been implicated for producing most binaries with periods below 7 d. We show that the geometry and orbital dynamics of these evolving triple-star systems are highly restrictive for a circumbinary planet, which is subject itself to Kozai modulation, on one hand, and can shield the two inner stars from their Kozai cycle and subsequent shrinking, on the other hand. Only small planets on wide and inclined orbits may form, survive and allow for the inner binary shrinkage. Those are difficult to detect.