Scheduling the NASA Deep Space Network with Deep Reinforcement Learning

Goh, Edwin; Venkataram, Hamsa Shwetha; Hoffmann, Mark; Johnston, Mark; Wilson, Brian

doi:10.1109/aero50100.2021.9438519

Cited by 4 publications

(8 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parallel developments on DSN scheduling at the Jet Propulsion Laboratory use Deep Reinforcement Learning [8]. Future work on this research could be to hybridize both approaches.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Space Network Scheduling via Mixed-Integer Linear Programming

et al. 2021

View full text Add to dashboard Cite

NASA's Deep Space Network (DSN) is a globally-spanning communications network responsible for supporting the interplanetary spacecraft missions of NASA and other international users. The DSN is a highly utilized asset, and the large demand for its' services makes the assignment of DSN resources a daunting computational problem. In this paper we study the DSN scheduling problem, which is the problem of assigning the DSN's limited resources to its users within a given time horizon. The DSN scheduling problem is oversubscribed, meaning that only a subset of the activities can be scheduled, and network operators must decide which activities to exclude from the schedule. We first formulate this challenging scheduling task as a Mixed-Integer Linear Programming (MILP) optimization problem. Next, we develop a sequential algorithm which solves the resulting MILP formulation to produce valid schedules for large-scale instances of the DSN scheduling problem. We use real world DSN data from week 44 of 2016 in order to evaluate our algorithm's performance. We find that given a fixed run time, our algorithm outperforms a simple implementation of our MILP model, generating a feasible schedule in which 17% more activities are scheduled by the algorithm than by the simple implementation. We design a non-MILP based heuristic to further validate our results. We find that our algorithm also outperforms this heuristic, scheduling 8% more activities and 20% more tracking time than the best results achieved by the non-MILP implementation.

show abstract

“…Parallel developments on DSN scheduling at the Jet Propulsion Laboratory use Deep Reinforcement Learning [8]. Future work on this research could be to hybridize both approaches.…”

Section: Discussionmentioning

confidence: 99%

“…7 Mars Reconnaissance Orbiter is a spacecraft that was designed to study the geology and climate of Mars, provide reconnaissance of future landing sites, and relay data from surface missions back to Earth. 8 Splitzer Space Telescope was designed to study the early universe in infrared light.…”

Section: Discussionmentioning

confidence: 99%

Deep Space Network Scheduling via Mixed-Integer Linear Programming

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Week 44 in 2016 (W44 2016) is an oversubscribed week where several algorithms tried to find the optimal schedule for, including [8] with Deep Reinforcement Learning (DeepRL) and [14] with Mixed-Integer Linear Programming (MILP). As shown in Table 2, W44 2016 consists of 14 resources, 284 activities, 1418 hours of requested tracking time, and 29 different missions.…”

Section: B Experimenting On Week 44 2016mentioning

confidence: 99%

“…In 2006, a genetic algorithm for DSN scheduling was proposed [6], and in 2008 a generalized differential evolutionary algorithm was investigated to find a Pareto optimal solution to satisfy multiple objective functions [7]. More recently, a deep reinforcement learning approach was proposed in which agents were trained to learn efficient strategies to generate feasible schedules based on DSN requests for a given week [8]. In addition, many novel solutions and improvements have been made to increase the operational efficiency of the DSN scheduling process [9]- [13].…”

Section: Introductionmentioning

confidence: 99%

Δ-MILP: Deep Space Network Scheduling via Mixed-Integer Linear Programming

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…More recently, a Squeaky Wheel optimization was used to study prioritization and oversubscribed scheduling of the DSN [3]. Recently, a deep reinforcement learning (RL) approach demonstrated that an agent could learn to outperform a random baseline through trial and error in a simulated DSN scheduling environment [11].…”

Section: A Dsn Schedulingmentioning

confidence: 99%

Deep Space Network Scheduling Using Quantum Annealing

Guillaume

Goh

Johnston

et al. 2022

IEEE Trans. Quantum Eng.

Self Cite

View full text Add to dashboard Cite

NASA's Deep Space Network (DSN) is responsible for communication and navigation for several NASA and international missions. The DSN comprises three complexes located in Goldstone (California, USA), Cambera (Australia) and Madrid (Spain). This distribution in longitude guarantees a full sky coverage. Each complex has one 70-meter and several 34-meter antennas. The network routinely serves a few dozens missions. The scheduling of the DSN is complex and involves humans interventions as well as automated solutions. In order to increase the level of automation, different computing paradigms have been explored. In this paper, we report on the quadratic unconstrained binary optimization (QUBO) formulation of the DSN scheduling, as well as a custom classical solver designed around some of the unique features of this scheduling problem. Thanks to a hybrid framework that extends the size of the problems that can be solved with a quantum annealer, we are able to generate a schedule from the QUBO formulation of the problem for one week's worth of user antenna requests, which represents the time period scheduled during operation. In other words, this work describes a real-world application of quantum annealing using real-world, operational data. We compare the resulting schedules' quality to solutions obtained using a mixed-integer linear programming formulation on a commercial solver. Our custom solver, based on a quantum-inspired optimization technique called Substochatic Monte-Carlo, while much faster in generating schedules, could only treat a subset of requests and hence we report its results independently.

show abstract

Scheduling the NASA Deep Space Network with Deep Reinforcement Learning

Cited by 4 publications

References 12 publications

Deep Space Network Scheduling via Mixed-Integer Linear Programming

Deep Space Network Scheduling via Mixed-Integer Linear Programming

Δ-MILP: Deep Space Network Scheduling via Mixed-Integer Linear Programming

Deep Space Network Scheduling Using Quantum Annealing

Contact Info

Product

Resources

About