Unsupervised Intuitive Physics from Visual Observations

Ehrhardt, Sébastien; Monszpart, Áron; Mitra, Niloy J.; Vedaldi, Andrea

doi:10.1007/978-3-030-20893-6_44

Cited by 17 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another method was proposed to obtain the property vector for each object from the video to predict the future trajectories of objects and infer the interpretable physical values, such as mass or rebound coefficients, using principle component analysis [4]. Some studies were conducted to predict the future movement of balls rolling in elliptical bowls [8,32], and others were conducted to predict future object states and future frames by inputting short video sequences [9,33] using the spatial transform network [34]. A study was also conducted to predict the future trajectories of a bouncing ball [5] by using a variational recurrent neural network [35].…”

Section: Learning Explicit Physicsmentioning

confidence: 99%

“…In another study, the future movement of objects in a video was predicted using physical expression as a prior [39]. In several studies, real data were used to train the models [8,32]. However, generation of large amounts of real data is costly and results in limited utility.…”

Section: Learning For Real Scenesmentioning

confidence: 99%

“…For example, studies have been conducted to infer the stability of stacked blocks from images [1][2][3] or to infer physical properties, such as mass or density, of objects from video sequences, where collisions occur between the objects [4,5]. Studies have also been conducted to predict future movements of objects [6,7] or predict physical properties and their changes [5,8,9] in a short video sequence. However, in most of these studies, only videos of clearly captured objects have been focused upon.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Dohae

Lee

2020

Sensors

View full text Add to dashboard Cite

We propose a deep neural network model that recognizes the position and velocity of a fast-moving object in a video sequence and predicts the object’s future motion. When filming a fast-moving subject using a regular camera rather than a super-high-speed camera, there is often severe motion blur, making it difficult to recognize the exact location and speed of the object in the video. Additionally, because the fast moving object usually moves rapidly out of the camera’s field of view, the number of captured frames used as input for future-motion predictions should be minimized. Our model can capture a short video sequence of two frames with a high-speed moving object as input, use motion blur as additional information to recognize the position and velocity of the object, and predict the video frame containing the future motion of the object. Experiments show that our model has significantly better performance than existing future-frame prediction models in determining the future position and velocity of an object in two physical scenarios where a fast-moving two-dimensional object appears.

show abstract

Section: Learning Explicit Physicsmentioning

confidence: 99%

Section: Learning For Real Scenesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Dohae

Lee

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Denil et al (2017) use reinforcement learning to pursue physical experiments. Ehrhardt et al (2017) use simulated motion sequences to teach a neural network to predict motion, where Sedaghat et al (2017) predict motion patterns in real videos. DâĂŹAgnolo & Wulzer (2019); De Simone & Jacques (2019) use neural networks to detect discrepancy between reference models and actual (synthetic) data.…”

Section: Introductionmentioning

confidence: 99%

Machines learn to infer stellar parameters just by looking at a large number of spectra

Sedaghat

Romaniello

Carrick

et al. 2021

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

Machine learning has been widely applied to clearly defined problems of astronomy and astrophysics. However, deep learning and its conceptual differences to classical machine learning have been largely overlooked in these fields. The broad hypothesis behind our work is that letting the abundant real astrophysical data speak for itself, with minimal supervision and no labels, can reveal interesting patterns that may facilitate discovery of novel physical relationships. Here, as the first step, we seek to interpret the representations a deep convolutional neural network chooses to learn, and find correlations in them with current physical understanding. We train an encoder–decoder architecture on the self-supervised auxiliary task of reconstruction to allow it to learn general representations without bias towards any specific task. By exerting weak disentanglement at the information bottleneck of the network, we implicitly enforce interpretability in the learned features. We develop two independent statistical and information-theoretical methods for finding the number of learned informative features, as well as measuring their true correlation with astrophysical validation labels. As a case study, we apply this method to a data set of ∼270 000 stellar spectra, each of which comprising ∼300 000 dimensions. We find that the network clearly assigns specific nodes to estimate (notions of) parameters such as radial velocity and effective temperature without being asked to do so, all in a completely physics-agnostic process. This supports the first part of our hypothesis. Moreover, we find with high confidence that there are ∼4 more independently informative dimensions that do not show a direct correlation with our validation parameters, presenting potential room for future studies.

show abstract

“…[46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64] for further discussions). Conversely, neural networks may also lead to new insights into how the human brain develops physical intuition from observations [65][66][67][68][69][70][71]. Very recently, physical variables were extracted in an unsupervised way from time series data of dynamical systems in Ref.…”

mentioning

confidence: 99%

Physics Insights from Neural Networks

Krenn

2020

Physics

View full text Add to dashboard Cite

Despite the success of neural networks at solving concrete physics problems, their use as a generalpurpose tool for scientific discovery is still in its infancy. Here, we approach this problem by modeling a neural network architecture after the human physical reasoning process, which has similarities to representation learning. This allows us to make progress towards the long-term goal of machine-assisted scientific discovery from experimental data without making prior assumptions about the system. We apply this method to toy examples and show that the network finds the physically relevant parameters, exploits conservation laws to make predictions, and can help to gain conceptual insights, e.g., Copernicus' conclusion that the solar system is heliocentric.

show abstract

Unsupervised Intuitive Physics from Visual Observations

Cited by 17 publications

References 27 publications

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Machines learn to infer stellar parameters just by looking at a large number of spectra

Physics Insights from Neural Networks

Contact Info

Product

Resources

About