Local heterogeneities in the structure and properties of metallic glasses have recently been predicted by computer simulations and also observed in experiments. These heterogeneities are important in understanding the stability and performance of metallic glasses. Progress has been made in measuring heterogeneities in elastic properties and local density down to length scales of less than 10 nm. In this review, we focus on studies of structural and mechanical heterogeneities with emphasis on those achieved by transmission electron microscopy which has an excellent spatial resolution, multifunctional detection modes, as well as in-situ testing capabilities. We argue that the next important step in understanding the behavior of metallic glasses lies in understanding the spatial and temporal correlations between the various structural and mechanical heterogeneities.