Differentiable simulation for system identification and visuomotor control

Last update: Dec 18, 2022

Related tags

Deep Learning gradsim

Overview

gradsim

gradSim: Differentiable simulation for system identification and visuomotor control

gradSim is a unified differentiable rendering and multiphysics framework that allows solving a range of control and parameter estimation tasks (rigid bodies, deformable solids, and cloth) directly from images/video. Our unified computation graph — spanning from the dynamics and through the rendering process — enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.

This repo is currently a stub. To be notified when code is released, plese "subscribe" to this issue.

Citing gradSim

For attribution in academic contexts, please cite this work as

Citation

Jatavallabhula and Macklin et al., "gradSim: Differentiable simulation for system identification and visuomotor control", ICLR 2021.

BibTeX citation

@article{gradsim,
  title   = {gradSim: Differentiable simulation for system identification and visuomotor control},
  author  = {Krishna Murthy Jatavallabhula and Miles Macklin and Florian Golemo and Vikram Voleti and Linda Petrini and Martin Weiss and Breandan Considine and Jerome Parent-Levesque and Kevin Xie and Kenny Erleben and Liam Paull and Florian Shkurti and Derek Nowrouzezahrai and Sanja Fidler},
  journal = {International Conference on Learning Representations (ICLR)},
  year    = {2021},
  url     = {https://openreview.net/forum?id=c_E8kFWfhp0},
  pdf     = {https://openreview.net/pdf?id=c_E8kFWfhp0},
}

You might also like...

Gesture-Volume-Control - This Python program can adjust the system's volume by using hand gestures

Gesture-Volume-Control This Python program can adjust the system's volume by usi

1 Dec 30, 2021

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN If you use this code for your research, please cite ou

41 Dec 8, 2022

Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

gtn_applications An applications library using GTN. Current examples include: Offline handwriting recognition Automatic speech recognition Installing

68 Dec 29, 2022

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

145 Dec 30, 2022

Fast, differentiable sorting and ranking in PyTorch

Torchsort Fast, differentiable sorting and ranking in PyTorch. Pure PyTorch implementation of Fast Differentiable Sorting and Ranking (Blondel et al.)

655 Jan 4, 2023

Joint Discriminative and Generative Learning for Person Re-identification. CVPR'19 (Oral)

Joint Discriminative and Generative Learning for Person Re-identification [Project] [Paper] [YouTube] [Bilibili] [Poster] [Supp] Joint Discriminative

1.2k Dec 30, 2022

Brax is a differentiable physics engine that simulates environments made up of rigid bodies, joints, and actuators

Brax is a differentiable physics engine that simulates environments made up of rigid bodies, joints, and actuators. It's also a suite of learning algorithms to train agents to operate in these environments (PPO, SAC, evolutionary strategy, and direct trajectory optimization are implemented).

1.5k Jan 2, 2023

Hardware accelerated, batchable and differentiable optimizers in JAX.

JAXopt Installation | Examples | References Hardware accelerated (GPU/TPU), batchable and differentiable optimizers in JAX. Installation JAXopt can be

621 Jan 8, 2023

Experiments with differentiable stacks and queues in PyTorch

Please use stacknn-core instead! StackNN This project implements differentiable stacks and queues in PyTorch. The data structures are implemented in s

141 Oct 6, 2022

Comments

AttributeError raised in multiple examples

Environment information: ubuntu:20.04 python:3.7 cuda:11.3 pytorch:1.10.1 kaolin:0.1.0

For example,when I run demo_cloth.py, an error raised as following: Using cached kernels Traceback (most recent call last): File "demo_cloth.py", line 81, in pos=builder.particle_positions[attach0] - (1.0, 0.0, 0.0), AttributeError: 'ModelBuilder' object has no attribute 'particle_positions'

I can't find "particle_positions" anywhere in "gradsim/gradsim/dflex" Please what should I do to solve this problem? By the way, what are your pytorch versions? I would like to try downgrading pytorch to a specific version to give it a try.

opened by wtichng 0

AttributeError raised in multiple examples

For example, when I run control_fem.py, error raised as following:

Error: 'Tuple' object has no attribute 'value' while transforming node <class 'ast.Subscript'> in func: eval_triangles at line: 41 col: 14: 
        f1 = qp * Dm[0, 0] + rp * Dm[1, 0]
Error: 'Tuple' object has no attribute 'value' while transforming node <class 'ast.BinOp'> in func: eval_triangles at line: 41 col: 9: 
        f1 = qp * Dm[0, 0] + rp * Dm[1, 0]
Error: 'Tuple' object has no attribute 'value' while transforming node <class 'ast.BinOp'> in func: eval_triangles at line: 41 col: 9: 
        f1 = qp * Dm[0, 0] + rp * Dm[1, 0]
Error: 'Tuple' object has no attribute 'value' while transforming node <class 'ast.Assign'> in func: eval_triangles at line: 41 col: 4: 
        f1 = qp * Dm[0, 0] + rp * Dm[1, 0]
Error: 'Tuple' object has no attribute 'value' while transforming node <class 'ast.FunctionDef'> in func: eval_triangles at line: 2 col: 0: 
    def eval_triangles(x: df.tensor(df.float3),
Traceback (most recent call last):
  File "/home/yueying/gradsim/examples/control_fem.py", line 12, in <module>
    from gradsim import dflex as df
  File "/home/yueying/gradsim/gradsim/dflex/__init__.py", line 34, in <module>
    kernel_init()
  File "/home/yueying/gradsim/gradsim/dflex/sim.py", line 92, in kernel_init
    kernels = df.compile()
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 1693, in compile
    adj = Adjoint(kernel.func, device='cuda')
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 628, in __init__
    adj.eval(adj.tree.body[0])
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 817, in eval
    out = adj.eval(f)
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 1029, in eval
    out = adj.eval(node.value)
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 915, in eval
    left = adj.eval(node.left)
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 916, in eval
    right = adj.eval(node.right)
  File "/home/yueying/gradsim/gradsim/dflex/adjoint.py", line 1013, in eval
    if isinstance(node.slice.value, ast.Num):
AttributeError: 'Tuple' object has no attribute 'value'

opened by 315930399 0

ImportError raised when I run hellogradsim.py

When I run hellogradsim.py, an error raised as following:

Traceback (most recent call last):
  File "/home/yueying/gradsim/examples/hellogradsim.py", line 10, in <module>
    from gradsim.renderutils import SoftRenderer, TriangleMesh
ImportError: cannot import name 'TriangleMesh' from 'gradsim.renderutils' (/home/yueying/gradsim/gradsim/renderutils/__init__.py)

opened by 315930399 3

Owner

GitHub

Differentiable simulation for system identification and visuomotor control

Related tags

Overview

gradsim

Citing gradSim

You might also like...

Gesture-Volume-Control - This Python program can adjust the system's volume by using hand gestures

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN

Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Fast, differentiable sorting and ranking in PyTorch

Joint Discriminative and Generative Learning for Person Re-identification. CVPR'19 (Oral)

Brax is a differentiable physics engine that simulates environments made up of rigid bodies, joints, and actuators

Hardware accelerated, batchable and differentiable optimizers in JAX.

Experiments with differentiable stacks and queues in PyTorch

Comments

AttributeError raised in multiple examples

AttributeError raised in multiple examples

ImportError raised when I run hellogradsim.py

Owner

Differentiable molecular simulation of proteins with a coarse-grained potential

A Python library for differentiable optimal control on accelerators.

Doosan robotic arm, simulation, control, visualization in Gazebo and ROS2 for Reinforcement Learning.

Autonomous Ground Vehicle Navigation and Control Simulation Examples in Python

🎃 Core identification module of AI powerful point reading system platform.

Builds a LoRa radio frequency fingerprint identification (RFFI) system based on deep learning techiniques

ROS-UGV-Control-Interface - Control interface which can be used in any UGV

Hand Gesture Volume Control is AIML based project which uses image processing to control the volume of your Computer.

Civsim is a basic civilisation simulation and modelling system built in Python 3.8.

Simulation of the solar system using various nummerical methods