Deep learning (DL) driven proactive resource allocation (RA) is a promising approach for the efficient management of network resources. However, DL models typically have a limitation that they do not capture the uncertainty due to the arrival of new unseen samples with a distribution different than the data distribution available at DL model-training time, leading to wrong resource usage predictions. To address this, we propose a confidence aware DL solution for the robust and reliable predictions of wireless channel utilization (CU) in shared spectrum bands. We utilize an encoder-decoder based Bayesian DL model to generate prediction intervals which capture the uncertainties in wireless CU. We use the CU predictions to design a novel metric score which in turn is utilized to make an adaptive RA algorithm. We show that a DL model capturing uncertainty in CU can achieve higher data rates for a wireless network. Both DL driven predictions and RA models are tested using synthetic data as well as real CU data collected in the University of Oulu. Using analytical and simulations results, we also study the stability of the proposed RA algorithm and show that it converges to a Nash equilibrium (NE). Our results reveal that the proposed algorithm converges to an NE under 2N iterations where N is the number of network access points.