Pytorch Implementation of Interaction Networks for Learning about Objects, Relations and Physics

Overview

Interaction-Network-Pytorch

Pytorch Implementraion of Interaction Networks for Learning about Objects, Relations and Physics.

Interaction Network is a powerful graph based framework for dynamic systems. It is able to simulate the physical trajectories of n-body, bouncing ball, and non-rigid string systems accurately over thousands of time steps, after training only on single step predictions.

Our results provide surprisingly strong evidence of IN’s ability to learn accurate physical simulations and generalize their training to novel systems with different numbers and configurations of objects and relations… Our interaction network implementation is the first learnable physics engine than can scale up to real-world problems, and is a promising template for new AI approaches to reasoning about other physical and mechanical systems, scene understanding, social perception, hierarchical planning, and analogical reasoning.

Also see nice blog post and keynote.

Thanks to jaesik817 tensorflow implementation for Physics Engine.

Issues
  • The model massively overfits

    The model massively overfits

    Hi,

    thanks a lot for releasing a 3rd party implementation of the paper. Neverthless, I'm afraid there is some problem with your code or at least the hyperparameters are not chosen well. This can be shown by looking at the validation error as follows:

    n_epoch = 100
    batches_per_epoch = 100
    
    data_test = gen(n_objects, True)
    
    losses = []
    losses_test = []
    for epoch in range(n_epoch):
        for _ in range(batches_per_epoch):
            objects, sender_relations, receiver_relations, relation_info, target = get_batch(data, 30)
            predicted = interaction_network(objects, sender_relations, receiver_relations, relation_info)
            loss = criterion(predicted, target)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            losses.append(np.sqrt(loss.data[0]))
    
        objects, sender_relations, receiver_relations, relation_info, target = get_batch(data_test, 30)
        predicted = interaction_network(objects, sender_relations, receiver_relations, relation_info)
        losses_test.append(np.sqrt(criterion(predicted, target).data[0]))
    
        clear_output(True)
        plt.figure(figsize=(20,5))
        plt.subplot(131)
        plt.title('Epoch %s RMS Train Error %s' % (epoch, np.mean(losses[-100:])))
        plt.plot(losses)
        plt.subplot(132)
        plt.title('Epoch %s RMS Test Error %s' % (epoch, np.mean(losses_test[-100:])))
        plt.plot(losses_test)        
        plt.show()
    

    I got train RMS 2.3 but validation RMS 209.6. Update: This is mostly because you train on just a single "scene", so that the network actually never sees any other masses and thus cannot generalise to those.

    opened by mys007 2
  • NameError: global name 'manimation' is not defined

    NameError: global name 'manimation' is not defined

    it seems lacking import at Line 82 in Physics_Engine.py

    opened by JudyYe 0
Owner
MetaRL HRL Memory Graphs
null
Official implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks"

DiscoGAN Official PyTorch implementation of Learning to Discover Cross-Domain Relations with Generative Adversarial Networks. Prerequisites Python 2.7

SK T-Brain 744 Jan 16, 2022
This's an implementation of deepmind Visual Interaction Networks paper using pytorch

Visual-Interaction-Networks An implementation of Deepmind visual interaction networks in Pytorch. Introduction For the purpose of understanding the ch

Mahmoud Gamal Salem 163 Nov 24, 2021
Official PyTorch implementation of "Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics".

Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics This repository is the official PyTorch implementation of "Physics-aware Differ

USC-Melady 42 Dec 29, 2021
git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction Code for the ECCV 2020 paper by Yiming Qian and Yasutaka Furukawa Getting

null 30 Dec 18, 2021
[NeurIPS 2021 Spotlight] Code for Learning to Compose Visual Relations

Learning to Compose Visual Relations This is the pytorch codebase for the NeurIPS 2021 Spotlight paper Learning to Compose Visual Relations. Demo Imag

Nan Liu 72 Dec 27, 2021
Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021)

TDEER (WIP) Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021) Overview TDEER is an e

Alipay 5 Dec 23, 2021
Code for the paper "Relation of the Relations: A New Formalization of the Relation Extraction Problem"

This repo contains the code for the EMNLP 2020 paper "Relation of the Relations: A New Paradigm of the Relation Extraction Problem" (Jin et al., 2020)

YYY 27 Dec 22, 2021
Must-read Papers on Physics-Informed Neural Networks.

PINNpapers Contributed by IDRL lab. Introduction Physics-Informed Neural Network (PINN) has achieved great success in scientific computing since 2017.

IDRL 108 Jan 15, 2022
Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs

PhyCRNet Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs Paper link: [ArXiv] By: Pu Ren, Chengping Rao, Yang

Pu Ren 9 Jan 10, 2022
PyTorch implementation for the visual prior component (i.e. perception module) of the Visually Grounded Physics Learner [Li et al., 2020].

VGPL-Visual-Prior PyTorch implementation for the visual prior component (i.e. perception module) of the Visually Grounded Physics Learner (VGPL). Give

Toru 7 Oct 20, 2021
A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery

PiSL A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery. Sun, F., Liu, Y. and Sun, H., 2021. Physics-informe

Fangzheng (Andy) Sun 5 Oct 5, 2021
CPF: Learning a Contact Potential Field to Model the Hand-object Interaction

Contact Potential Field This repo contains model, demo, and test codes of our paper: CPF: Learning a Contact Potential Field to Model the Hand-object

Lixin YANG 72 Jan 12, 2022
Populating 3D Scenes by Learning Human-Scene Interaction https://posa.is.tue.mpg.de/

Populating 3D Scenes by Learning Human-Scene Interaction [Project Page] [Paper] License Software Copyright License for non-commercial scientific resea

Mohamed Hassan 64 Jan 6, 2022
Graph Self-Attention Network for Learning Spatial-Temporal Interaction Representation in Autonomous Driving

GSAN Introduction Code for paper GSAN: Graph Self-Attention Network for Learning Spatial-Temporal Interaction Representation in Autonomous Driving, wh

YE Luyao 5 Oct 5, 2021
PaddleRobotics is an open-source algorithm library for robots based on Paddle, including open-source parts such as human-robot interaction, complex motion control, environment perception, SLAM positioning, and navigation.

简体中文 | English PaddleRobotics paddleRobotics是基于paddle的机器人开源算法库集,包括人机交互、复杂运动控制、环境感知、slam定位导航等开源算法部分。 人机交互 主动多模交互技术TFVT-HRI 主动多模交互技术是通过视觉、语音、触摸传感器等输入机器人

null 135 Jan 14, 2022
[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

Rex Cheng 252 Jan 11, 2022
Synthesizing Long-Term 3D Human Motion and Interaction in 3D in CVPR2021

Long-term-Motion-in-3D-Scenes This is an implementation of the CVPR'21 paper "Synthesizing Long-Term 3D Human Motion and Interaction in 3D". Please ch

Jiashun Wang 58 Jan 10, 2022
The GitHub repository for the paper: “Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction“.

SCINet This is the original PyTorch implementation of the following work: Time Series is a Special Sequence: Forecasting with Sample Convolution and I

null 119 Jan 13, 2022
Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2

CoaDTI Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2 Abstract Environment The test was conducted i

Layne_Huang 1 Dec 22, 2021