Learning-aided Stochastic Network Optimization with Imperfect State Prediction

Abstract

Abstract-We investigate the problem of stochastic network optimization in the presence of imperfect state prediction and non-stationarity. Based on a novel distribution-accuracy curve prediction model, we develop the predictive learning-aided control (PLC) algorithm, which jointly utilizes historic and predicted network state information for decision making. PLC is an online algorithm that requires zero a-prior system statistical information, and consists of three key components, namely sequential distribution estimation and change detection, dual learning, and online queue-based control.Specifically, we show that PLC simultaneously achieves good long-term performance, short-term queue size reduction, accurate change detection, and fast algorithm convergence. In particular, for stationary networks, PLC achieves a near-optimal [O( ), O(log(1/ )2 )] utility-delay tradeoff. For non-stationary networks,) time, where ew is the prediction accuracy and a = Θ(1) > 0 is a constant (the Backpressue algorithm [1] requires an O( −2 ) length for the same utility performance with a larger backlog). Moreover, PLC detects distribution change O(w) slots faster with high probability (w is the prediction size) and achieves an O(min( −1+c/2 , ew/ ) + log 2 (1/ )) convergence time, which is faster than Backpressure and other algorithms. Our results demonstrate that state prediction (even imperfect) can help (i) achieve faster detection and convergence, and (ii) obtain better utility-delay tradeoffs. They also quantify the benefits of prediction in four important performance metrics, i.e., utility (efficiency), delay (quality-of-service), detection (robustness), and convergence (adaptability), and provide new insight for joint prediction, learning and optimization in stochastic networks.

Timely-Throughput Optimal Scheduling With Prediction

“…[15] shows that proactive scheduling can effectively reduce queueing delay in stochastic single-queue systems. [16] considers how network state prediction can be incorporated into algorithm design. [17] and [18] focus on understanding the cost saving aspect of proactive scheduling based on demand prediction.…”