Spatially resolved electron energy‐loss spectroscopy (EELS) is rapidly developing into a unique and powerful tool to characterize internal interfaces. Because atomic column resolved Z‐contrast imaging can be performed simultaneously with EELS in the scanning transmission electron microscope, this combination allows the atomic structure to be correlated with the electronic structure, and thus the local properties of interfaces or defects can be determined directly. However, the ability to characterize interfaces and defects at that level requires not only high spatial resolution but also the exact knowledge of the beam location, from where the spectrum is obtained. Here we discuss several examples progressing from cases where the limitation in spatial resolution is given by the microscopes or the nature of the sample, to one example of impurity atoms at a grain boundary, which show intensity and fine structure changes from atomic column to atomic column. Such data can be interpreted as changes in valence of the impurity, depending on its exact site in the boundary plane. Analysis of this nature is a valuable first step in understanding the macroscopic structural, optical and electronic properties of materials.