With the explosive growth of mobile data demand, the fifth generation (5G) mobile network would exploit the enormous amount of spectrum in the millimeter wave (mmWave) bands to greatly increase communication capacity. There are fundamental differences between mmWave communications and existing other communication systems, in terms of high propagation loss, directivity, and sensitivity to blockage. These characteristics of mmWave communications pose several challenges to fully exploit the potential of mmWave communications, including integrated circuits and system design, interference management, spatial reuse, anti-blockage, and dynamics control. To address these challenges, we carry out a survey of existing solutions and standards, and propose design guidelines in architectures and protocols for mmWave communications. We also discuss the potential applications of mmWave communications in the 5G network, including the small cell access, the cellular access, and the wireless backhaul. Finally, we discuss relevant open research issues including the new physical layer technology, software-defined network architecture, measurements of network state information, efficient control mechanisms, and heterogeneous networking, which should be further investigated to facilitate the deployment of mmWave communication systems in the future 5G networks.