Spatial gaps correspond to the projection in position space of the gaps of a periodic structure whose envelope varies spatially. They can be easily generated in cold atomic physics using finitesize optical lattice, and provide a new kind of tunnel barriers which can be used as a versatile tool for quantum devices. We present in detail different theoretical methods to quantitatively describe these systems, and show how they can be used to realize in one dimension matter wave Fabry-Perot cavities. We also provide experimental and numerical results that demonstrate the interest of spatial gaps structures for phase space engineering. We then generalize the concept of spatial gaps in two dimensions and show that this enables to design multiply connected cavities which generate a quantum dot structure for atoms or allow to construct curved wave guides for matter waves. At last, we demonstrate that modulating in time the amplitude of the periodic structure offers a wide variety of possible atom manipulations including the control of the scattering of an incoming wave packet, the loading of cavities delimited by spatial gaps, their coupling by multiphonon processes or the realization of a tunable source of atoms. This large range of possibilities offered by space and time engineering of optical lattices demonstrates the flexibility of such band gap structures for matter wave control, quantum simulators and atomtronics.