5MART: A 5G SMART Scheduling Framework for Optimizing QoS Through Reinforcement Learning

Abstract

Full bibliographic details must be given when referring to, or quoting from full items including the author's name, the title of the work, publication details where relevant (place, publisher, date), pagination, and for theses or dissertations the awarding institution, the degree type awarded, and the date of the award.

A Machine Learning Resource Allocation Solution to Improve Video Quality in Remote Education

“…If the solutions also consider performing network resource allocation in a heterogeneous environment, there is a need for innovative approaches such as use of Machine Learning (ML). As an example, the ML-based OFDMA scheduling approach proposed in [23] can accommodate up to 50% more connections in terms of 360-degree and traditional video content when compared to state-of-the-art multi-class schedulers.…”

“…5, at the scheduler level, data packets belonging to video traffic class with the lowest bit rates are waiting to be transmitted in data queues with priority 1 while packets corresponding to learners with the highest bit rates are awaiting in data queues with priority P , where P is the total number of video classes. In general, the OFDMA packet scheduler is designed to work in two stages [23]: a) Time Domain Prioritization (TDP) where learners with more stringent QoS requirements are prioritized over other learners requesting video content with more relaxed QoS budget; and b) the Frequency Domain Prioritization (FDP), where the preselected group of learners are competing in getting the best radio resources to receive the requested video services. Both stages are iteratively performed at each Transmission Time Interval (TTI) to respect the QoS requirements for all P video classes.…”

“…where ρ t+1 is the reward at TTI t+1 and E[•] is the expectation showing that s(t) ∈ S is considered as random such that P[s(t) = s] > 0 and P[a(t) = a] > 0 holds for all possible actions a ∈ A. By employing the transition function from (3) and splitting the action in master and slave decisions, the reward function can be written as follows [23]:…”

“…The instantaneous controllable state becomesȳ = τ c (y), whereȳ ∈S C p and τ c can be a set of statistical functions employed to extract the relevant data from a vector of L p elements. We adopt the same statistical model proposed in [23], where the controllable elements {x p , x p , q p } can be modeled as normally distributed variables and then, τ c employs the mean and standard deviation functions based on the maximum likelihood estimators. As result, the dimension ofȳ p is reduced from L p • (2O p + 1) to 2 • (2O p + 1), that depends only on O p number of objectives that must be met to keep high service quality for the delivered educational content.…”

“…where η ∈ [0, 1] is the learning rate and δ p = F T p (•; θ p ) − F p (•; θ p ) is the Time Difference (TD) error calculated as the difference between the target value and the actual estimate of the neural network. The TD error is back-propagated through the neural networks to update the weights [23].…”