Traffic accidents caused by drowsy drivers represent a crucial threat to public safety. Recent statistics show that drowsy drivers cause an estimated 15.5% of fatal accidents. With the widespread use of mobile devices and roadside units, these accidents can be significantly prevented using a drowsiness detection solution. While several solutions were proposed in the literature, they all fall short of presenting a distributed architecture that can answer the needs of these applications without breaching the driver's privacy. This paper proposes a two-stage Driver Drowsiness Detection System using smart edge computing. Mobile devices in the car are used to capture and analyze the current condition of the drivers without sharing their data. The smart edge is deployed as a decision-maker where the drowsiness is confirmed when the information about the driver status received from the mobile client and the observed car path match. Our approach relies on a) a distributed edge architecture that has two levels of hierarchy, namely the Main Edge Node (MEN) and Local Edge Node (LEN), to better manage the area of interest and b) a data fusion offloading strategy that considers: 1) local detection of driver drowsiness through facial expressions using CNN model, 2) global detection of car path through acceleration readings using YoLov5 algorithm, and finally, 3) a two-layer LSTM algorithm for drowsiness detection based on the local and the global detection. The proposed framework achieves drowsiness detection with an average accuracy of 97.7%.