Testing of deep learning models is challenging due to the excessive number and complexity of the computations involved. As a result, test data selection is performed manually and in an ad hoc way. This raises the question of how we can automatically select candidate data to test deep learning models. Recent research has focused on defining metrics to measure the thoroughness of a test suite and to rely on such metrics to guide the generation of new tests. However, the problem of selecting/prioritising test inputs (e.g., to be labelled manually by humans) remains open. In this article, we perform an in-depth empirical comparison of a set of test selection metrics based on the notion of model uncertainty (model confidence on specific inputs). Intuitively, the more uncertain we are about a candidate sample, the more likely it is that this sample triggers a misclassification. Similarly, we hypothesise that the samples for which we are the most uncertain are the most informative and should be used in priority to improve the model by retraining. We evaluate these metrics on five models and three widely used image classification problems involving real and artificial (adversarial) data produced by five generation algorithms. We show that uncertainty-based metrics have a strong ability to identify misclassified inputs, being three times stronger than surprise adequacy and outperforming coverage-related metrics. We also show that these metrics lead to faster improvement in classification accuracy during retraining: up to two times faster than random selection and other state-of-the-art metrics on all models we considered.

Abstract-In this paper, an intelligent probing method for interference constraint learning is proposed to allow a centralized Cognitive Radio Network (CRN) to access the frequency band of a Primary User (PU) operating based on an Adaptive Coding and Modulation (ACM) protocol. The main idea is that the CRN probes the PU and subsequently applies a Modulation and Coding Classification (MCC) technique to acquire the Modulation and Coding scheme (MCS) of the PU. This feedback is an implicit channel state information (CSI) of the PU link, indicating how harmful the probing induced interference is. The intelligence of this sequential probing process lies on the selection of the power levels of the Secondary Users (SUs) which aims to minimize the number of probing attempts, a clearly Active Learning (AL) procedure, and consequently the overall PU QoS degradation. The enhancement introduced in this work is that we incorporate the probability of each feedback being correct into this intelligent probing mechanism by using a univariate Bayesian Nonparametric AL method, the Probabilistic Bisection Algorithm (PBA). An adaptation of the PBA is implemented for higher dimensions and its effectiveness as an uncertainty driven AL method is demonstrated through numerical simulations.

No abstract

Abstract-A key concept suggested for 5G networks is spectrum sharing within the context of Cognitive Communications (CC). This efficient spectrum usage has been explored intensively the last years. In this paper, a mechanism is proposed to allow a cognitive user, also called Secondary User (SU), to access the frequency band of a Primary User (PU) operating based on an Adaptive Coding and Modulation (ACM) protocol. The Spectrum Sensing (SS) technique used considers Higher Order Statistical (HOS) features of the signal and log-likelihood ratios (LLRs) of the code syndromes in order to constantly monitor the modulation and coding scheme (MODCOD) of the PU respectively. Once the Modulation and Coding Classification (MCC) is completed, a Power Control (PC) scheme is enabled. The SU can attempt to access the frequency band of the PU and increase its transmitting power until it causes a change of the PU's transmission scheme due to interference. When the SU detects the change of the PU's MODCOD, then it reduces its transmitting power to a lower level so as to regulate the induced interference. The proposed blind Adaptive Power Control (APC) algorithm converges without any interference channel information to the aforementioned interference limit and guarantees the preservation of the PU link throughput.

Edge-caching is recognized as an efficient technique for future cellular networks to improve network capacity and user-perceived quality of experience. To enhance the performance of caching systems, designing an accurate content request prediction algorithm plays an important role. In this paper, we develop a flexible model, a Poisson regressor based on a Gaussian process, for the content request distribution. The first important advantage of the proposed model is that it encourages the already existing or seen contents with similar features to be correlated in the feature space and therefore it acts as a regularizer for the estimation. Second, it allows to predict the popularities of newly-added or unseen contents whose statistical data is not available in advance. In order to learn the model parameters, which yield the Poisson arrival rates or alternatively the content popularities, we invoke the Bayesian approach which is robust against over-fitting. However, the resulting posterior distribution is analytically intractable to compute. To tackle this, we apply a Markov Chain Monte Carlo (MCMC) method to approximate this distribution which is also asymptotically exact. Nevertheless, the MCMC is computationally demanding especially when the number of contents is large. Thus, we employ the Variational Bayes (VB) method as an alternative low complexity solution. More specifically, the VB method addresses the approximation of the posterior distribution through an optimization problem. Subsequently, we present a fast block-coordinate descent algorithm to solve this optimization problem. Finally, extensive simulation results both on synthetic and real-world datasets are provided to show the accuracy of our prediction algorithm and the cache hit ratio (CHR) gain compared to existing methods from the literature.