Millimeter wave (mmWave) communications have emerged as one of the most promising options to vastly increase wireless data rates due to the high bandwidth they offer. Given the high path loss at mmWave frequencies, such systems require directional antennas to achieve a good communication range. The communicating devices thus need to align the beam directions of their mmWave antennas. Due to the high penetration loss, the paths between the antennas also need to be free of blocking obstacles. This makes efficient and reliable operation of mmWave networks in dynamic environments very challenging. At the same time, the directionality reduces interference and allows to scale these networks to much higher access point and device densities. In this article, we discuss the above challenges and present techniques that allow mmWave networks to scale to high-density deployments, to adapt to dynamic and mobile environments, and to consistently achieve high data rates. This includes learning the environment to find different propagation paths, reacting timely to channel impairments such as blockage, and integrating mmWave networks with networks operating at lower-frequency for robustness. A key ingredient to enable these forms of adaptivity is the use of location information. Such mechanisms then turn a collection of very-high-speed but brittle mmWave links into an efficient, low-latency, and reliable network.