A major component of the estimation and correction of starlight at very high contrasts is the creation of a dark hole: a region in the vicinity of the core of the stellar point spread function (PSF) where speckles in the PSF wings have been greatly attenuated, up to a factor of 10 10 for the imaging of terrestrial exoplanets. At these very high contrasts, removing these speckles requires distinguishing between light from the stellar PSF scattered by instrument imperfections, which may be partially corrected across a broad band using deformable mirrors in the system, from light from other sources which generally may not. These other sources may be external or internal to the instrument (e.g. planets, exozodiacal light), but in either case, their distinguishing characteristic is their inability to interfere coherently with the PSF. In the following we discuss the estimation, structure, and expected origin of this "incoherent" signal, primarily in the context of a series of experiments made with a linear band-limited mask in Jan-Mar 2013. We find that the "incoherent" signal at moderate contrasts is largely estimation error of the coherent signal, while at very high contrasts it represents a true floor which is stable over week-timescales.