With the explosive growth of mobile data demand, there has been an increasing interest in deploying small cells of higher frequency bands underlying the conventional homogeneous macrocell network, which is usually referred to as heterogeneous cellular networks, to significantly boost the overall network capacity. With vast amounts of spectrum available in the millimeter wave (mmWave) band, small cells at mmWave frequencies are able to provide multi-gigabit access data rates, while the wireless backhaul in the mmWave band is emerging as a cost-effective solution to provide high backhaul capacity to connect access points (APs) of the small cells. In order to operate the mobile network optimally, it is necessary to jointly design the radio access and backhaul networks. Meanwhile, direct transmissions between devices should also be considered to improve system performance and enhance the user experience. In this paper, we propose a joint transmission scheduling scheme for the radio access and backhaul of small cells in the mmWave band, termed D2DMAC, where a path selection criterion is designed to enable device-to-device transmissions for performance improvement. In D2DMAC, a concurrent transmission scheduling algorithm is proposed to fully exploit spatial reuse in mmWave networks. Through extensive simulations under various traffic patterns and user deployments, we demonstrate D2DMAC achieves near-optimal performance in some cases, and outperforms other protocols significantly in terms of delay and throughput. Furthermore, we also analyze the impact of path selection on the performance improvement of D2DMAC under different selected parameters.