Events of high dust loading are extreme meteorological phenomena with important climate and health implications. Therefore, early forecasting is critical for mitigating their adverse effects. Dust modeling is a long-standing challenge due to the multiscale nature of the governing meteorological dynamics and the complex coupling between atmospheric particles and the underlying atmospheric flow patterns. While physics-based numerical modeling is commonly being used, we propose a meteorological-based deep multi-task learning approach for forecasting dust events. Our approach consists of forecasting the local PM10 (primary task) measured in situ, and simultaneously to predict the satellite-based regional PM10 (auxiliary task); thus, leveraging valuable information from a correlated task. We use 18 years of regional meteorological data to train a neural forecast model for dust events in Israel. Twenty-four hours before the dust event, the model can detect 76% of the events with even higher predictability of winter and spring events. Further analysis shows that local dynamics drive most misclassified events, meaning that the coherent driving meteorology in the region holds a predictive skill. Further, we use machine-learning interpretability methods to reveal the meteorological patterns the model has learned, thus highlighting the important features that govern dust events in the Middle East, being primarily lower-tropospheric winds, and Aerosol Optical Depth.