Thank you for providing your code.

I've been trying to run the franka_reacher.py demo, but I'm getting a surprising behavior from the target (red) mug. It falls through the table, and then progressively slides to a corner of the environment (see attached video).

Is that the expected behavior? I'm guessing it is not, especially because the behavior of the robot is significantly different when setting the vis_ee_target variable to False. In that case, the robot tries to reach an (invisible) target in front of it, which looks much more like a proper demo.

Any help with this weird behavior?

https://user-images.githubusercontent.com/114128734/193214376-abbba2b0-ab62-455f-a406-dd6ffbfe38db.mp4

I am trying to run the example file python franka_reacher.py. But on running the red mug falls through the base and floats below. (Screenshot attached). Kindly let me know how to fix this please.

Hi,

great work with this repository. Really appreciated the fact that you put it open-source. I encountered a problem when running the script due to a potential bug in mpc/control.

After the installation according to your instruction, everything works smooth until I encounter the following error:

self.sample_shape = torch.Size([self.num_nonzero_particles - 2], device=self.tensor_args['device'])
TypeError: tuple() takes at most 1 argument (2 given)

I removed the second argument in two locations: https://github.com/NVlabs/storm/blob/cf056514659cab9c5c7ac9d850eae7c56a660726/storm_kit/mpc/control/olgaussian_mpc.py#L112 https://github.com/NVlabs/storm/blob/cf056514659cab9c5c7ac9d850eae7c56a660726/storm_kit/mpc/control/sample_libs.py#L315

With these modifications, everything works smoothly. It might be related to the torch version. I have installed torch v.1.10.0 (automatically through your setup file)

Let me know if you have run into this problem, or tell me how I could fix it on my side.

Best, Max.

Hi,

I'm running the example franka_reacher.py on ubuntu 18 with

• NVIDIA GeForce MX130, 2GB of memory.
• Cuda toolkit 11.2

What can I do to fix this memory error that shows up sometimes?

Process Process-1:
Traceback (most recent call last):
  File "/home/rgap/miniconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/rgap/miniconda3/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rgap/LIBRARIES/storm/storm_kit/mpc/utils/mpc_process_wrapper.py", line 212, in optimize_process
    controller = torch.load(control_string)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 857, in persistent_load
    load_tensor(data_type, size, key, _maybe_decode_ascii(location))
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 846, in load_tensor
    loaded_storages[key] = restore_location(storage, location)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 175, in default_restore_location
    result = fn(storage, location)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/serialization.py", line 157, in _cuda_deserialize
    return obj.cuda(device)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/_utils.py", line 79, in _cuda
    return new_type(self.size()).copy_(self, non_blocking)
  File "/home/rgap/.virtualenvs/exp/lib/python3.8/site-packages/torch/cuda/__init__.py", line 528, in _lazy_new
    return super(_CudaBase, cls).__new__(cls, *args, **kwargs)

RuntimeError: CUDA error: out of memory

CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
['95.704', '1.288', '0.742'] 0.138 0.020
python3.8: /opt/conda/conda-bld/magma-cuda102_1583546904148/work/interface_cuda/interface.cpp:897: void magma_queue_create_from_cuda_internal(magma_device_t, cudaStream_t, cublasHandle_t, cusparseHandle_t, magma_queue**, const char*, const char*, int): Assertion `queue->dAarray__ != __null' failed.
/home/rgap/miniconda3/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Hi, thank you for your excellent job! I want to design my own MPC controller based on your work and test it on the Franka robot. But I met some problems. Franka robot needs a real-time Ubuntu kernel, and "storm" needs an Nvidia GPU. I have installed the Nvidia driver 465 on the Ubuntu 5.4.10-rt5 kernel. But when I inputted Nvidia-smi in the terminal, it shows that it couldn't communicate with the Nvidia driver. I found "Nvidia driver cannot work on a real-time ubuntu kernel" on Google. So I want to know how to test your work with a Franka robot. Thank you!

Can you please explain the different collision constraints:

1. primitive_collisions: I couldn't find any info on this
2. robot_self_collision: I am assuming this is for not colliding with the arm links
3. scene_collision: This is for not colliding with environment obstacles using SceneCollisionNet.
4. voxel_collision: I couldn't find any info on this

It would be really helpful if you can explain what primitive Collisions and Voxel Collisions are for. Thank you.

My team is working on implementing STORM on a UR16e robot. We defined the collision spheres to the best of our abilities following the example provided for the Franka Panda robot and using the stl files for the UR16e. The robot still collides with primitives. I have attached our robot collision sphere definition file and the mpc definition file. This video shows an example of the collision occurring.

After speaking to Mohak we are trying to debug this issue by visualizing the spheres utilizing the w_batch_link_spheres object and update_batch_robot_collision_objs method in the class RobotSphereCollision in robot.py.

Any further guidance or an existing method/code would be appreciated.

ur16e_collision.txt ur16e_reacher_collision.txt