Fig. 1. Reinforcement learning over a geometric fabric layer yields safe, high-performance manipulation behavior for a highly-actuated hand. The learned behavior switches between two-, three-, and four-fingered grasps during prehensile manipulation. Videos at https://dextreme.org/fgp.html.

Second-order differential equations define smooth system behavior. In general, there is no guarantee that a system will behave well when forced by a potential function, but in some cases they do and may exhibit smooth optimization properties such as convergence to a local minimum of the potential. Such a property is desirable and inherently linked to asymptotic stability. This paper presents a comprehensive theory of optimization fabrics which are second-order differential equations that encode nominal behaviors on a space and are guaranteed to optimize when forced by a potential function. Optimization fabrics, or fabrics for short, can encode commonalities among optimization problems that reflect the structure of the space itself, enabling smooth optimization processes to intelligently navigate each problem even when the potential function is simple and relatively naïve. Importantly, optimization over a fabric is asymptotically stable, so optimization fabrics constitute a building block for stable system design.

No abstract

No abstract

Most physical systems have dynamics functions that are just a nuisance to policies. Torque policies, for instance, usually have to effectively invert the natural classical mechanical dynamics to get their job done. Because of this, we often use controllers to make things easier on policies. For instance, inverse dynamics controllers wipe out the physical dynamics so the policy starts from a clean slate. That makes learning easier, but still the policy needs to learn everything about the problem, including aspects of a solution which are common to many other problems, such as how to make the end-effector move in a straight line, how to avoid joints and self collisions, how to avoid obstacles, etc. Over the past few years it's become standard to formulate learning not in C-space, but in end-effector space and use controllers such as Operational Space Control (OSC) to capture some of these commonalities. These controllers, whether inverse dynamics or OSC, reshape the natural dynamics of the system into a different second-order dynamical system whose behavior is more useful. And the trend is, the more useful behavior we can pack into these reshaped systems, the easier it is to learn policies.However, OSC is from the 80's, and captures only straight line end-effector motion. There's a lot more behavior we could and should be packing into these systems. Earlier work [15,16,19] developed a theory that generalized these ideas and constructed a broad and flexible class of second-order dynamical systems which was simultaneously expressive enough to capture substantial behavior (such as that listed above), and maintained the types of stability properties that make OSC and controllers like it a good foundation for policy design and learning. This paper, motivated by the empirical success of the types of fabrics used in [20], reformulates the theory of fabrics into a form that's more general and easier to apply to policy learning problems. We focus on the stability properties that make fabrics a good foundation for policy synthesis. Fabrics create a fundamentally stable medium within which a policy can operate; they influence the system's behavior without preventing it from achieving tasks within its constraints. When a fabrics is geometric (path consistent) we can interpret the fabric as forming a road network of paths that the system wants to follow at constant speed absent a forcing policy, giving geometric intuition to its role as a prior. The policy operating over the geometric fabric acts to modulate speed and steers the system from one road to the next as it accomplishes its task.We reformulate the theory of fabrics here rigorously and develop theoretical results characterizing system behavior and illuminating how to design these systems, while also emphasizing intuition throughout.