Symbolic pregression: Discovering physical laws from distorted video

Udrescu, Silviu-Marian; Tegmark, Max

doi:10.1103/physreve.103.043307

Cited by 31 publications

(20 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This could be achieved by joint learning of the underlying representation and equations of motion, e.g., by minimizing the prediction error and complexity of the equations of motion. Such approaches have been shown capable of recovering physical laws in Cartesian coordinates from warped video footage 36 and it would be interesting to extend this to complex biological systems, where the underlying laws are less clear. Modeling of cell growth in terms of generalized shape coordinates is an area of active research, with one promising model balancing dissipative, mechanical, and active forces 37 .…”

Section: Discussionmentioning

confidence: 99%

Physics-informed deep learning characterizes morphodynamics of Asian soybean rust disease

et al. 2021

View full text Add to dashboard Cite

Medicines and agricultural biocides are often discovered using large phenotypic screens across hundreds of compounds, where visible effects of whole organisms are compared to gauge efficacy and possible modes of action. However, such analysis is often limited to human-defined and static features. Here, we introduce a novel framework that can characterize shape changes (morphodynamics) for cell-drug interactions directly from images, and use it to interpret perturbed development of Phakopsora pachyrhizi, the Asian soybean rust crop pathogen. We describe population development over a 2D space of shapes (morphospace) using two models with condition-dependent parameters: a top-down Fokker-Planck model of diffusive development over Waddington-type landscapes, and a bottom-up model of tip growth. We discover a variety of landscapes, describing phenotype transitions during growth, and identify possible perturbations in the tip growth machinery that cause this variation. This demonstrates a widely-applicable integration of unsupervised learning and biophysical modeling.

show abstract

Section: Discussionmentioning

confidence: 99%

Physics-informed deep learning characterizes morphodynamics of Asian soybean rust disease

et al. 2021

View full text Add to dashboard Cite

show abstract

“…(1) An initial attempt 1 at an algorithm that is able to recognize both physical governing equations and governing parameters from videos. Previous work [8], [9] can either recognize governing equations or the parameters, but not both. (2) We test the algorithm on both synthetic data and real data.…”

Section: Introductionmentioning

confidence: 99%

On Learning Mechanical Laws of Motion From Video Using Neural Networks

Chari

Zhou

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In computer vision, physics plays an important role in several applications. In this work, we teach a machine to detect the mechanical laws of motion of physical objects using video, and show how the results can be useful for computer vision tasks. We assume no prior knowledge of physics, beyond a temporal stream of bounding boxes. The problem is very difficult because a machine must learn not only a governing equation (e.g. projectile motion) but also the existence of governing parameters (e.g. velocities). We evaluate our ability to represent the physical laws of motion in video, such as the movement of a projectile or circular motion, in both real and constructed videos. These elementary tasks have textbook governing equations and enable ground truth verification of our approach. To establish the importance of the proposed method, we show a real-world use case in the domain of object tracking in confounding scenes, where existing state-of-the-art algorithms fail. Incorporating physics into computer vision not only serves the purpose of curiosity-driven research, but also provides an inductive bias for computer vision applications like object tracking.INDEX TERMS Mechanical laws of motion, Video analysis, Object tracking.

show abstract

“…For instance (Figure C), H-bond clustering is a phenomenon that is at the basis to explain protein stability across species, time, and type of protein. Can we develop tools that can discover such relationships directly from data , and learn how all possible interactions in a physical system distill to a few key functional relationships, such as captured in the details of an interatomic potential (Figure A)? Such relationships can then infer many material properties, including how the nonlinear nature of chemical bonding facilitates supersonic fracture. − Once discovered, we can use these concepts to explain phenomena in a variety of materials, achieving generalization about how interactions between building blocks of a system yield emergent system-level properties.…”

Section: Introductionmentioning

confidence: 99%

Multiscale Modeling at the Interface of Molecular Mechanics and Natural Language through Attention Neural Networks

Buehler

2022

Acc. Chem. Res.

View full text Add to dashboard Cite

Conspectus Humans are continually bombarded with massive amounts of data. To deal with this influx of information, we use the concept of attention in order to perceive the most relevant input from vision, hearing, touch, and others. Thereby, the complex ensemble of signals is used to generate output by querying the processed data in appropriate ways. Attention is also the hallmark of the development of scientific theories, where we elucidate which parts of a problem are critical, often expressed through differential equations. In this Account we review the emergence of attention-based neural networks as a class of approaches that offer many opportunities to describe materials across scales and modalities, including how universal building blocks interact to yield a set of material properties. In fact, the self-assembly of hierarchical, structurally complex, and multifunctional biomaterials remains a grand challenge in modeling, theory, and experiment. Expanding from the process by which material building blocks physically interact to form a type of material, in this Account we view self-assembly as both the functional emergence of properties from interacting building blocks as well as the physical process by which elementary building blocks interact and yield structure and, thereby, functions. This perspective, integrated through the theory of materiomics, allows us to solve multiscale problems with a first-principles-based computational approach based on attention-based neural networks that transform information to feature to property while providing a flexible modeling approach that can integrate theory, simulation, and experiment. Since these models are based on a natural language framework, they offer various benefits including incorporation of general domain knowledge via general-purpose pretraining, which can be accomplished without labeled data or large amounts of lower-quality data. Pretrained models then offer a general-purpose platform that can be fine-tuned to adapt these models to make specific predictions, often with relatively little labeled data. The transferrable power of the language-based modeling approach realizes a neural olog description, where mathematical categorization is learned by multiheaded attention, without domain knowledge in its formulation. It can hence be applied to a range of complex modeling taskssuch as physical field predictions, molecular properties, or structure predictions, all using an identical formulation. This offers a complementary modeling approach that is already finding numerous applications, with great potential to solve complex assembly problems, enabling us to learn, build, and utilize functional categorization of how building blocks yield a range of material functions. In this Account, we demonstrate the approach in various application areas, including protein secondary structure prediction and prediction of normal-mode frequencies as well as predicting mechanical fields near cracks. Unifying these diverse problem areas is the building block approach, where the...

show abstract

Symbolic pregression: Discovering physical laws from distorted video

Cited by 31 publications

References 49 publications

Physics-informed deep learning characterizes morphodynamics of Asian soybean rust disease

Physics-informed deep learning characterizes morphodynamics of Asian soybean rust disease

On Learning Mechanical Laws of Motion From Video Using Neural Networks

Multiscale Modeling at the Interface of Molecular Mechanics and Natural Language through Attention Neural Networks

Contact Info

Product

Resources

About