Digital holographic microscopy (DHM) as a label-free quantitative imaging tool has been widely used to investigate the morphology of living cells dynamically. In the offaxis DHM, the spatial filtering in the frequency spectrum of the hologram is vital to the quality of the reconstructed images. In this paper, we propose an adaptive spatial filtering approach based on convolutional neural networks (CNN) to automatically extracts the optimal shape of frequency components. For achieving robust and precise recognition performance, the net model is trained by using the tens of thousands of frequency spectrums with a variety of specimens and imaging conditions. The experimental results demonstrate that the trained network produce an adaptive spatial filtering window which can accurately select the frequency components of the object term and eliminate the frequency components of the interference terms, especially the coherent noise that overlaps with the object term in the spatial frequency domain. We find that the proposed approach has a fast, robust, and outstanding frequency filtering capability without any manual intervention and initial input parameters compared to previous techniques. Furthermore, the applicability of the proposed method in off-axis DHM for dynamic analysis is demonstrated by real-time monitoring the morphologic changes of living MLO-Y4 cells that are constantly subject to Fluid Shear Stress (FSS).