Drift counteraction optimal control for deterministic systems and enhancing convergence of value iteration

Zidek, Robert A. E.; Kolmanovsky, Ilya

doi:10.1016/j.automatica.2017.06.015

Cited by 6 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, according to the sequence of identification states s ∅ , s brand , s model, s all and seq, the corresponding actions in the sequence seq of each state are selected in turn and repeatedly, and the optimal multi-protocol probe sequence of a type-known IoT device is obtained. According to the related works [25,26], it can be proved that the algorithm has convergence, that is, the optimal protocol probe sequence obtained is unique. Due to space limitation, the proof process is not given here.…”

Section: Value Iteration Algorithmmentioning

confidence: 99%

A Time-efficient Multi-Protocol Probe Scheme for Fine-grain IoT Device Identification

Chen

et al. 2020

Sensors

View full text Add to dashboard Cite

Internet of Things (IoT) devices connected to the Internet are exploding, which poses a significant threat for their management and security protection. IoT device identification is a prerequisite for discovering, monitoring, and protecting these devices. Although we can identify the device type easily through grabbing protocol banner information, both brand and model of different types of device are various and diverse. We should therefore utilize multi-protocol probes to improve the fineness of device identification and obtain the corresponding brand and model. However, it is still a challenge to balance between the multi-protocol probe overhead and the identification fineness. To solve this problem, we proposed a time-efficient multi-protocol probe scheme for fine-grain devices identification. We first adopted the concept of reinforcement learning to model the banner-based device identification process into a Markov decision process (MDP). Through the value iteration algorithm, an optimal multi-protocol probe sequence is generated for a type-known IoT device, and then the optimal multi-protocol probes sequence segment is extracted based on the gain threshold of identification accuracy. We took 132,835 webcams as the sample data to experiment. The experimental results showed that our optimal multi-protocol probes sequence segment could reduce the identification time of webcams’ brand and model by 50.76% and achieve the identification accuracy of 90.5% and 92.3% respectively. In addition, we demonstrated that our time-efficient optimal multi-protocol probe scheme could also significantly improve the identification efficiency of other IoT devices, such as routers and printers.

show abstract

Section: Value Iteration Algorithmmentioning

confidence: 99%

A Time-efficient Multi-Protocol Probe Scheme for Fine-grain IoT Device Identification

Chen

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…However, it is generally difficult to solve an HJB equation either analytically or numerically, especially considering that it may not admit a smooth solution. As a result, recent advances in DCOC have been in a discrete-time setting [7][8][9][10][11][12][13][14][15], which is more computationally tractable. Apart from computational benefits, a discrete-time formulation also leads to control solutions that are easily implementable with digital micro-controllers.…”

Section: Introductionmentioning

confidence: 99%

“…Apart from computational benefits, a discrete-time formulation also leads to control solutions that are easily implementable with digital micro-controllers. For discrete-time systems, DCOC approaches based on dynamic programming (DP) [7][8][9][10][11][12] and mixed-integer programming (MIP) [13][14][15] have been developed. However, both DP-based and MIP-based approaches are faced with computational challenges: The former can treat DCOC problems with general yield functional [7] but suffer from the curse of dimensionality [16];…”

Section: Introductionmentioning

confidence: 99%

Continuous Optimization-Based Drift Counteraction Optimal Control: A Spacecraft Attitude Control Case Study

Tang¹,

Li²,

Zidek³

et al. 2021

Preprint

View full text Add to dashboard Cite

“…In [10], problem (5) was solved using dynamic programming techniques. Due to the curse of dimensionality, this approach is limited to lower-dimensional problems.…”

mentioning

confidence: 99%

Spacecraft Drift Counteraction Optimal Control: Open-Loop and Receding Horizon Solutions

Zidek

Kolmanovsky

Bemporad

2018

Journal of Guidance, Control, and Dynamics

Self Cite

View full text Add to dashboard Cite

Drift counteraction optimal control for deterministic systems and enhancing convergence of value iteration

Cited by 6 publications

References 15 publications

A Time-efficient Multi-Protocol Probe Scheme for Fine-grain IoT Device Identification

A Time-efficient Multi-Protocol Probe Scheme for Fine-grain IoT Device Identification

Continuous Optimization-Based Drift Counteraction Optimal Control: A Spacecraft Attitude Control Case Study

Spacecraft Drift Counteraction Optimal Control: Open-Loop and Receding Horizon Solutions

Contact Info

Product

Resources

About