CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels

Xiao, Yongqian; Xu, Xiaodong; Lin, Qianli

doi:10.48550/arxiv.2102.10205

Cited by 3 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Firstly, [18] uses DMD and rescaled DMD (rDMD) for image processing. Also relating to images, [81] used Deterministic and Convolutional Koopman Networks (DCKNet and CKNet, respectively) to predict a suitable trajectory from a provided topography to solve the standard Mountain Car Problem. This may have relevance to energy-limited adaptive cruise control applications in the automotive category.…”

Section: B Theoretical Issues With Potential Applications To Smart Mo...mentioning

confidence: 99%

Vehicular Applications of Koopman Operator Theory—A Survey

2023

View full text Add to dashboard Cite

Koopman operator theory has proven to be a promising approach to nonlinear system identification and global linearization. For nearly a century, there had been no efficient means of calculating the Koopman operator for applied engineering purposes. The introduction of a recent computationally efficient method in the context of fluid dynamics, which is based on the system dynamics decomposition to a set of normal modes in descending order, has overcome this long-lasting computational obstacle. The purely data-driven nature of Koopman operators holds the promise of capturing unknown and complex dynamics for reduced-order model generation and system identification, through which the rich machinery of linear control techniques can be utilized. Given the ongoing development of this research area and the many existing open problems in the fields of smart mobility and vehicle engineering, a survey of techniques and open challenges of applying Koopman operator theory to this vibrant area is warranted. This review focuses on the various solutions of the Koopman operator which have emerged in recent years, particularly those focusing on mobility applications, ranging from characterization and component-level control operations to vehicle performance and fleet management. Moreover, this comprehensive review of over 100 research papers highlights the breadth of ways Koopman operator theory has been applied to various vehicular applications with a detailed categorization of the applied Koopman operator-based algorithm type. Furthermore, this review paper discusses theoretical aspects of Koopman operator theory that have been largely neglected by the smart mobility and vehicle engineering community and yet have large potential for contributing to solving open problems in these areas.

show abstract

Section: B Theoretical Issues With Potential Applications To Smart Mo...mentioning

confidence: 99%

Vehicular Applications of Koopman Operator Theory—A Survey

2023

View full text Add to dashboard Cite

show abstract

“…Many successful applications of the Koopman-DMD approach have been reported. These include fluid mechanics [13], power systems [12], optimal control [19], and computer vision [20], [21]. In robotics, too, the Koopman-DMD approach has made significant contributions in the last several years.…”

Section: Introductionmentioning

confidence: 99%

Global, Unified Representation of Heterogenous Robot Dynamics Using Composition Operators: A Koopman Direct Encoding Method

Asada

2023

IEEE/ASME Trans. Mechatron.

View full text Add to dashboard Cite

The dynamic complexity of robots and mechatronic systems often pertains to the hybrid nature of dynamics, where governing equations consist of heterogenous equations that are switched depending on the state of the system. Legged robots and manipulator robots experience contact-noncontact discrete transitions, causing switching of governing equations. Analysis of these systems has been a challenge due to the lack of a global, unified model that is amenable to analysis of the global behaviors. Composition operator theory has the potential to provide a global, unified representation by converting them to linear dynamical systems in a lifted space. This article presents a method for encoding nonlinear heterogenous dynamics into a high-dimensional space of observables in the form of Koopman operator. First, a new formula is established for representing the Koopman operator in a Hilbert space by using inner products of observable functions and their composition with the governing state transition function. This formula, called direct encoding, allows for converting a class of heterogenous systems directly to a global, unified linear model. Unlike prevalent data-driven methods, where results can vary depending on numerical data, the proposed method is globally valid, not requiring numerical simulation of the original dynamics. A simple example validates the theoretical results, and the method is applied to a multicable suspension system.

show abstract

DDK: A Deep Koopman Approach for Dynamics Modeling and Trajectory Tracking of Autonomous Vehicles

Xiao¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous driving has attracted lots of attention in recent years. An accurate vehicle dynamics is important for autonomous driving techniques, e.g. trajectory prediction, motion planning, and control of trajectory tracking. Although previous works have made some results, the strong nonlinearity, precision, and interpretability of dynamics for autonomous vehicles are open problems worth being studied. In this paper, the approach based on the Koopman operator named deep direct Koopman (DDK) is proposed to identify the model of the autonomous vehicle and the identified model is a linear time-invariant (LTI) version, which is convenient for motion planning and controller design. In the approach, the Koopman eigenvalues and system matrix are considered as trainable tensors with the original states of the autonomous vehicle being concatenated to a part of the Koopman eigenfunctions so that a physically interpretable subsystem can be extracted from the identified latent dynamics. Subsequently, the process of the identification model is trained under the proposed method based on the dataset which consists of about 60km of data collected with a real electric SUV while the effectiveness of the identified model is validated. Meanwhile, a high-fidelity vehicle dynamics is identified in CarSim with DDK, and then, a linear model predictive control (MPC) called DDK-MPC integrating DDK is designed to validate the performance for the control of trajectory tracking. Simulation results illustrate that the model of the nonlinear vehicle dynamics can be identified effectively via the proposed method and that excellent tracking performance can be obtained with the identified model under DDK-MPC.

show abstract

CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels

Cited by 3 publications

References 26 publications

Vehicular Applications of Koopman Operator Theory—A Survey

Vehicular Applications of Koopman Operator Theory—A Survey

Global, Unified Representation of Heterogenous Robot Dynamics Using Composition Operators: A Koopman Direct Encoding Method

DDK: A Deep Koopman Approach for Dynamics Modeling and Trajectory Tracking of Autonomous Vehicles

Contact Info

Product

Resources

About