[NeurIPS 2021] PyTorch Code for Accelerating Robotic Reinforcement Learning with Parameterized Action Primitives

Related tags

Deep Learning reinforcement-learning deep-learning robotics artificial-intelligence primitives
Overview

Robot Action Primitives (RAPS)

This repository is the official implementation of Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives (RAPS).

[Project Website]

Murtaza Dalal, Deepak Pathak*, Ruslan Salakhutdinov*
(* equal advising)

CMU

alt text

If you find this work useful in your research, please cite:

@inproceedings{dalal2021raps,
    Author = {Dalal, Murtaza and Pathak, Deepak and
              Salakhutdinov, Ruslan},
    Title = {Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives},
    Booktitle = {NeurIPS},
    Year = {2021}
}

Requirements

To install dependencies, please run the following commands:

sudo apt-get update
sudo apt-get install curl \
    git \
    libgl1-mesa-dev \
    libgl1-mesa-glx \
    libglew-dev \
    libosmesa6-dev \
    software-properties-common \
    net-tools \
    unzip \
    vim \
    virtualenv \
    wget \
    xpra \
    xserver-xorg-dev
sudo apt-get install libglfw3-dev libgles2-mesa-dev patchelf
sudo mkdir /usr/lib/nvidia-000

Please add the following to your bashrc:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.mujoco/mujoco200/bin
export MUJOCO_GL='egl'
export MKL_THREADING_LAYER=GNU
export D4RL_SUPPRESS_IMPORT_ERROR='1'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia-000

To install python requirements:

conda create -n raps python=3.7
conda activate raps
./setup_python_env.sh <absolute path to raps>

Training and Evaluation

Kitchen

Prior to running any experiments, make sure to run cd /path/to/raps/rlkit

single task env names:

  • microwave
  • kettle
  • slide_cabinet
  • hinge_cabinet
  • light_switch
  • top_left_burner

multi task env names:

  • microwave_kettle_light_top_left_burner //Sequential Multi Task 1
  • hinge_slide_bottom_left_burner_light //Sequential Multi Task 2

To train RAPS with Dreamer on any single task kitchen environment, run:

python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train RAPS with Dreamer on the multi task kitchen environments, run:

python experiments/kitchen/dreamer/dreamer_v2_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train Raw Actions with Dreamer on any kitchen environment

python experiments/kitchen/dreamer/dreamer_v2_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train RAPS with RAD on any single task kitchen environment

python experiments/kitchen/rad/rad_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train RAPS with RAD on any multi task kitchen environment

python experiments/kitchen/rad/rad_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train Raw Actions with RAD on any kitchen environment

python experiments/kitchen/rad/rad_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train RAPS with PPO on any single task kitchen environment

python experiments/kitchen/ppo/ppo_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train RAPS with PPO on any multi task kitchen environment

python experiments/kitchen/ppo/ppo_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train Raw Actions with PPO on any kitchen environment

python experiments/kitchen/ppo/ppo_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>

Metaworld

single task env names

  • drawer-close-v2
  • soccer-v2
  • peg-unplug-side-v2
  • sweep-into-v2
  • assembly-v2
  • disassemble-v2

To train RAPS with Dreamer on any metaworld environment

python experiments/metaworld/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>

To train Raw Actions with Dreamer on any metaworld environment

python experiments/metaworld/dreamer/dreamer_v2_single_task_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>

Robosuite

To train RAPS with Dreamer on an Robosuite Lift

python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_lift.py --mode here_no_doodad --exp_prefix <>

To train Raw Actions with Dreamer on an Robosuite Lift

python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_lift.py --mode here_no_doodad --exp_prefix <>

To train RAPS with Dreamer on an Robosuite Door

python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py --mode here_no_doodad --exp_prefix <>

To train Raw Actions with Dreamer on an Robosuite Door

python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_door.py --mode here_no_doodad --exp_prefix <>

Learning Curve visualization

cd /path/to/raps/rlkit
python ../viskit/viskit/frontend.py data/<exp_prefix> //open localhost:5000 to view
Comments
  • Errors when attempting to train

    Errors when attempting to train

    Hello and once again thanks for the amazing work!

    Context

    So far, I've managed to install the repo but running tests has proven difficult.

    An issue I found early on was that running a sample training command like mentioned in the README didn't work as the program seemed to be unable to remember the environment passed by --env. This issue is mentioned in #1, and for the sake of consistency I will use the same command moving forward in these issues: $ python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env microwave

    With a bit of debugging I think I've managed to solve it, although this issue is not limited to dreamer_v2_single_task_primitives.py : https://github.com/mihdalal/raps/blob/23c7d952c9d5e4db6fd96b9a56c1c0373d2bd121/rlkit/experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py#L117 should actually be search_space = {"env_name": [args.exp_prefix]}.

    Using sed for this replacement shows the following to be all the affected files (I can make a PR later if it seems like this issue is not exclusive to me):

    experiments/kitchen/dreamer/dreamer_v2_multi_task_primitives.py
experiments/kitchen/dreamer/dreamer_v2_raw_actions.py
experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py
experiments/kitchen/dreamer/dreamer_v2_vices.py
experiments/kitchen/ppo/ppo_raw_actions.py
experiments/kitchen/ppo/ppo_vices.py
experiments/metaworld/dreamer/dreamer_v2_single_task_primitives.py
experiments/metaworld/dreamer/dreamer_v2_single_task_raw_actions.py
experiments/metaworld/dreamer/dreamer_v2_single_task_vices.py

    Issue

    With this problem out of the way, however, here is the real focus of this issue - when running the command again after solving the aforementioned problem, this is the output error (with similar results for other kitchen training scripts):

    $ python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env microwave                            
duplicate key ignored: env_name
2022-05-14 14:51:12.216333 WEST | Variant:
2022-05-14 14:51:12.216544 WEST | {
  "env_name": "microwave",
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 250,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 30,
    "num_train_loops_per_epoch": 40,
    "num_trains_per_train_loop": 10
  },
  "use_raw_actions": false,
  "env_suite": "kitchen",
  "pass_render_kwargs": true,
  "save_video": true,
  "env_kwargs": {
    "dense": false,
    "image_obs": true,
    "action_scale": 1.4,
    "use_workspace_limits": true,
    "control_mode": "primitives",
    "usage_kwargs": {
      "use_dm_backend": true,
      "use_raw_action_wrappers": false,
      "use_image_obs": true,
      "max_path_length": 5,
      "unflatten_images": false
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 5,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "seed": "94249",
  "exp_id": "0",
  "exp_prefix": "05-14-microwave",
  "instance_type": "None",
  "exp_name": "05-14-microwave_2022_05_14_14_51_12_0000--s-94249"
}
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-1:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-2:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
Process SpawnProcess-4:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
Traceback (most recent call last):
  File "experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py", line 139, in <module>
    exp_id=exp_id,
  File "~/xstar-raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "~/xstar-raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

    As I am not too well acquainted with the internals of MuJoCo, this would be very difficult for me to debug. I would very thankful if you could provide some insight in how to solve this error.

    opened by feup-jmc 4
  • Erros when testing

    Erros when testing

    Hello and thanks for your sharing of this amazing work! I'm trying to run the code you recommended python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --env microwave however, i found an error which is not mentioned previously in other issues

    2022-06-08 17:39:37.331497 CST | Variant:
2022-06-08 17:39:37.332199 CST | {
  "env_name": "microwave",
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 250,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 30,
    "num_train_loops_per_epoch": 40,
    "num_trains_per_train_loop": 10
  },
  "use_raw_actions": false,
  "env_suite": "kitchen",
  "pass_render_kwargs": true,
  "save_video": true,
  "env_kwargs": {
    "dense": false,
    "image_obs": true,
    "action_scale": 1.4,
    "use_workspace_limits": true,
    "control_mode": "primitives",
    "usage_kwargs": {
      "use_dm_backend": true,
      "use_raw_action_wrappers": false,
      "use_image_obs": true,
      "max_path_length": 5,
      "unflatten_images": false
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 5,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "seed": "23764",
  "exp_id": "0",
  "exp_prefix": "06-08-test",
  "instance_type": "None",
  "exp_name": "06-08-test_2022_06_08_17_39_37_0000--s-23764"
}
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Process SpawnProcess-2:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Process SpawnProcess-5:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Traceback (most recent call last):
  File "experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py", line 139, in <module>
    exp_id=exp_id,
  File "/home/lwt/github_test/raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

    I searched google for answers to solve the MjData problem, but nothing was found. I wonder if you can give me some advice, thanks for your selfless assistance!

    opened by papercut-linkin 3
  • Contradictory dependencies make building impossible

    Contradictory dependencies make building impossible

    Hello, and thanks for the cool repo on the paper. I would like to use RAPS in another project but due to the contradictory dependencies in https://github.com/mihdalal/raps/blob/ec6d54abd2e24128cf1d9f433ceb08276d6dae73/viskit/setup.py#L7 and https://github.com/mihdalal/raps/blob/ec6d54abd2e24128cf1d9f433ceb08276d6dae73/requirements.txt#L4 https://github.com/mihdalal/raps/blob/ec6d54abd2e24128cf1d9f433ceb08276d6dae73/requirements.txt#L12 https://github.com/mihdalal/raps/blob/ec6d54abd2e24128cf1d9f433ceb08276d6dae73/requirements.txt#L21 with the rest of the repo, I get the following errors when setting it up:

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-gpu 2.8.0 requires numpy>=1.20, but you have numpy 1.16.0 which is incompatible.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.5.0 which is incompatible.
pywavelets 1.3.0 requires numpy>=1.17.3, but you have numpy 1.16.0 which is incompatible.
pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.16.0 which is incompatible.
metaworld 0.0.0 requires numpy>=1.18, but you have numpy 1.16.0 which is incompatible.

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.5.0 which is incompatible.
robosuite 1.2.0 requires mujoco-py==2.0.2.9, but you have mujoco-py 2.0.2.13 which is incompatible.
viskit 0.1 requires matplotlib==2.0.2, but you have matplotlib 3.3.4 which is incompatible.
viskit 0.1 requires numpy==1.16.0, but you have numpy 1.21.6 which is incompatible.

    What should I do? Just ignore the version specification on the conflicting dependencies?

    opened by feup-jmc 3
  • Aggressive swapping in robosuite training

    Aggressive swapping in robosuite training

    Hello, and once again thanks for the great work on this repo!

    This is not so much an issue with RAPS errors but rather something I've observed when training RAPS. Unlike the other suites included with RAPS, robosuite has very high disk usage during training. I am unsure whether this is an isolated issue, but there is a constant writing of data to swap of between 0 and 400 MB/s (with some peaks as high as 1200MB/s). This can be quite damaging to an SSD in the long-term (like in my setup), given the limited number of writes characteristic of these drives.

    image

    Is there something that can be done about this?

    opened by feup-jmc 2
  • Bump lxml from 4.6.5 to 4.9.1 in /d4rl/scripts/brac

    Bump lxml from 4.6.5 to 4.9.1 in /d4rl/scripts/brac

    Bumps lxml from 4.6.5 to 4.9.1.

    Changelog

    Sourced from lxml's changelog.

    4.9.1 (2022-07-01)

    Bugs fixed

    • A crash was resolved when using iterwalk() (or canonicalize()) after parsing certain incorrect input. Note that iterwalk() can crash on valid input parsed with the same parser after failing to parse the incorrect input.

    4.9.0 (2022-06-01)

    Bugs fixed

    • GH#341: The mixin inheritance order in lxml.html was corrected. Patch by xmo-odoo.

    Other changes

    • Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.

    • Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35 (libxml2 2.9.12+ and libxslt 1.1.34 on Windows).

    • GH#343: Windows-AArch64 build support in Visual Studio. Patch by Steve Dower.

    4.8.0 (2022-02-17)

    Features added

    • GH#337: Path-like objects are now supported throughout the API instead of just strings. Patch by Henning Janssen.

    • The ElementMaker now supports QName values as tags, which always override the default namespace of the factory.

    Bugs fixed

    • GH#338: In lxml.objectify, the XSI float annotation "nan" and "inf" were spelled in lower case, whereas XML Schema datatypes define them as "NaN" and "INF" respectively.

    ... (truncated)

    Commits
    • d01872c Prevent parse failure in new test from leaking into later test runs.
    • d65e632 Prepare release of lxml 4.9.1.
    • 86368e9 Fix a crash when incorrect parser input occurs together with usages of iterwa...
    • 50c2764 Delete unused Travis CI config and reference in docs (GH-345)
    • 8f0bf2d Try to speed up the musllinux AArch64 build by splitting the different CPytho...
    • b9f7074 Remove debug print from test.
    • b224e0f Try to install 'xz' in wheel builds, if available, since it's now needed to e...
    • 897ebfa Update macOS deployment target version from 10.14 to 10.15 since 10.14 starts...
    • 853c9e9 Prepare release of 4.9.0.
    • d3f77e6 Add a test for https://bugs.launchpad.net/lxml/+bug/1965070 leaving out the a...
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump lxml from 4.4.1 to 4.6.5 in /d4rl/scripts/brac

    Bump lxml from 4.4.1 to 4.6.5 in /d4rl/scripts/brac

    Bumps lxml from 4.4.1 to 4.6.5.

    Changelog

    Sourced from lxml's changelog.

    4.6.5 (2021-12-12)

    Bugs fixed

    • A vulnerability (GHSL-2021-1038) in the HTML cleaner allowed sneaking script content through SVG images (CVE-2021-43818).

    • A vulnerability (GHSL-2021-1037) in the HTML cleaner allowed sneaking script content through CSS imports and other crafted constructs (CVE-2021-43818).

    4.6.4 (2021-11-01)

    Features added

    • GH#317: A new property system_url was added to DTD entities. Patch by Thirdegree.

    • GH#314: The STATIC_* variables in setup.py can now be passed via env vars. Patch by Isaac Jurado.

    4.6.3 (2021-03-21)

    Bugs fixed

    • A vulnerability (CVE-2021-28957) was discovered in the HTML Cleaner by Kevin Chung, which allowed JavaScript to pass through. The cleaner now removes the HTML5 formaction attribute.

    4.6.2 (2020-11-26)

    Bugs fixed

    • A vulnerability (CVE-2020-27783) was discovered in the HTML Cleaner by Yaniv Nizry, which allowed JavaScript to pass through. The cleaner now removes more sneaky "style" content.

    4.6.1 (2020-10-18)

    ... (truncated)

    Commits
    • a9611ba Fix a test in Py2.
    • a3eacbc Prepare release of 4.6.5.
    • b7ea687 Update changelog.
    • 69a7473 Cleaner: cover some more cases where scripts could sneak through in specially...
    • 54d2985 Fix condition in test decorator.
    • 4b220b5 Use the non-depcrecated TextTestResult instead of _TextTestResult (GH-333)
    • d85c6de Exclude a test when using the macOS system libraries because it fails with li...
    • cd4bec9 Add macOS-M1 as wheel build platform.
    • fd0d471 Install automake and libtool in macOS build to be able to install the latest ...
    • f233023 Cleaner: Remove SVG image data URLs since they can embed script content.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump tensorflow from 1.15.2 to 2.5.3 in /d4rl/scripts/brac

    Bump tensorflow from 1.15.2 to 2.5.3 in /d4rl/scripts/brac

    Bumps tensorflow from 1.15.2 to 2.5.3.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.5.3

    Release 2.5.3

    Note: This is the last release in the 2.5 series.

    This releases introduces several vulnerability fixes:

    • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
    • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
    • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
    • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
    • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
    • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
    • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
    • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
    • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
    • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
    • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
    • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
    • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
    • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
    • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
    • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
    • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
    • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
    • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
    • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
    • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
    • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)
    • Fixes an integer overflow in TFLite array creation (CVE-2022-23558)
    • Fixes an integer overflow in TFLite (CVE-2022-23559)
    • Fixes a dangerous OOB write in TFLite (CVE-2022-23561)
    • Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)
    • Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)
    • Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)
    • Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)
    • Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)
    • Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)
    • Fixes a heap OOB write in Grappler (CVE-2022-23566)
    • Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)
    • Fixes an unitialized variable access in AssignOp (CVE-2022-23573)
    • Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)
    • Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)
    • Fixes a null dereference in GetInitOp (CVE-2022-23577)
    • Fixes a memory leak when a graph node is invalid (CVE-2022-23578)
    • Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)
    • Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)
    • Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)
    • Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)
    • Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.5.3

    This releases introduces several vulnerability fixes:

    • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
    • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
    • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
    • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
    • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
    • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
    • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
    • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
    • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
    • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
    • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
    • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
    • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
    • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
    • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
    • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
    • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
    • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
    • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
    • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
    • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
    • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)

    ... (truncated)

    Commits
    • 959e9b2 Merge pull request #54213 from tensorflow/fix-sanity-on-r2.5
    • d05fcbc Fix sanity build
    • f2526a0 Merge pull request #54205 from tensorflow/disable-flaky-tests-on-r2.5
    • a5f94df Disable flaky test
    • 7babe52 Merge pull request #54201 from tensorflow/cherrypick-510ae18200d0a4fad797c0bf...
    • 0e5d378 Set Env Variable to override Setuptools new behavior
    • fdd4195 Merge pull request #54176 from tensorflow-jenkins/relnotes-2.5.3-6805
    • 4083165 Update RELEASE.md
    • a2bb7f1 Merge pull request #54185 from tensorflow/cherrypick-d437dec4d549fc30f9b85c75...
    • 5777ea3 Update third_party/icu/workspace.bzl
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Connection reset by peer

    Connection reset by peer

    Good day,

    After trying the suggested solutions in the other two issues on the same topic, when running

    python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py --mode here_no_doodad --exp_prefix test

    I still get

    2022-12-06 19:05:46.506113 CET | Variant:
2022-12-06 19:05:46.506515 CET | {
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 1000,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 60,
    "num_train_loops_per_epoch": 20,
    "num_trains_per_train_loop": 20
  },
  "env_name": "Door",
  "use_raw_actions": false,
  "env_suite": "robosuite",
  "env_kwargs": {
    "robots": "Panda",
    "has_renderer": false,
    "has_offscreen_renderer": false,
    "use_camera_obs": false,
    "camera_heights": 64,
    "camera_widths": 64,
    "controller_configs": {
      "type": "OSC_POSE",
      "input_max": 1,
      "input_min": -1,
      "output_max": [
        0.1,
        0.1,
        0.1,
        0.5,
        0.5,
        0.5
      ],
      "output_min": [
        -0.1,
        -0.1,
        -0.1,
        -0.5,
        -0.5,
        -0.5
      ],
      "kp": 150,
      "damping_ratio": 1,
      "impedance_mode": "fixed",
      "kp_limits": [
        0,
        300
      ],
      "damping_ratio_limits": [
        0,
        10
      ],
      "position_limits": null,
      "orientation_limits": null,
      "uncouple_pos_ori": true,
      "control_delta": true,
      "interpolation": null,
      "ramp_ratio": 0.2
    },
    "horizon": 5,
    "control_freq": 20,
    "reward_shaping": false,
    "reset_action_space_kwargs": {
      "control_mode": "primitives",
      "action_scale": 1,
      "max_path_length": 5,
      "workspace_low": [
        -0.3,
        -0.4,
        0.8
      ],
      "workspace_high": [
        0.0,
        0.2,
        1.3
      ],
      "go_to_pose_iterations": 40,
      "camera_settings": {
        "distance": 0.9532481338137215,
        "lookat": [
          -0.30494697896254724,
          -0.4394507345054956,
          0.9492024838769221
        ],
        "azimuth": -125.859375,
        "elevation": -29.062499923165888
      }
    },
    "usage_kwargs": {
      "use_dm_backend": true,
      "max_path_length": 5
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 10,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "pass_render_kwargs": true,
  "save_video": true,
  "seed": "31079",
  "exp_id": "0",
  "exp_prefix": "12-06-test",
  "instance_type": "None",
  "exp_name": "12-06-test_2022_12_06_19_05_46_0000--s-31079"
}
Process SpawnProcess-7:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-8:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-10:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-2:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-3:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-6:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-4:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-5:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-9:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Traceback (most recent call last):
  File "/home/dve/raps/rlkit/experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py", line 167, in <module>
    run_experiment(
  File "/home/dve/raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "/home/dve/raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 255, in recv
    buf = self._recv_bytes()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes
    buf = self._recv(4)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

    Note that my installed packages are:

    Package                  Version       
------------------------ ----------------- 
a2c-ppo-acktr            0.0.1     
absl-py                  1.3.0
awscli                   1.27.23
boto                     2.49.0
boto3                    1.26.23
botocore                 1.29.23
certifi                  2022.9.24
cffi                     1.15.1
charset-normalizer       2.1.1
click                    8.0.1
cloudpickle              1.6.0
colorama                 0.4.4
contourpy                1.0.6
cycler                   0.11.0
Cython                   0.29.32
d4rl                     1.1               
dm-control               1.0.0
dm-env                   1.5
dm-tree                  0.1.7
docutils                 0.16
doodad                   0.2.1.dev0        /home/dve/raps/doodad
fasteners                0.18
Flask                    2.1.2
fonttools                4.38.0
future                   0.18.2
glfw                     2.5.5
gtimer                   1.0.0b5
gym                      0.18.3
gym-notices              0.0.8
h5py                     3.7.0
idna                     3.4
imageio                  2.9.0
importlib-metadata       4.13.0
itsdangerous             2.1.2
Jinja2                   3.1.2
jmespath                 1.0.1
kiwisolver               1.4.4
labmaze                  1.0.6
llvmlite                 0.36.0
lxml                     4.9.1
MarkupSafe               2.1.1
matplotlib               3.3.4
metaworld                0.0.0    
mjrl                     1.0.0
mujoco-py                2.0.2.13
networkx                 2.8.8
numba                    0.53.1
numpy                    1.21.6
numpy-quaternion         2021.4.5.14.42.35
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
opencv-python            4.5.2.52
packaging                21.3
pandas                   1.5.2
Pillow                   8.2.0
pip                      22.2.2
plotly                   4.0.0
protobuf                 4.21.10
pyasn1                   0.4.8
pybullet                 3.2.5
pycparser                2.21
pyglet                   1.5.15
PyOpenGL                 3.1.6
pyparsing                2.4.7
python-dateutil          2.8.1
pytz                     2022.6
PyWavelets               1.4.1
PyYAML                   5.4.1
rad                      1.0.0          
requests                 2.28.1
retrying                 1.3.4
rlkit                    0.2.1.dev0   
robosuite                1.2.0          
rsa                      4.7.2
s3transfer               0.6.0
scikit-image             0.17.2
scipy                    1.9.3
setuptools               65.5.0
six                      1.16.0
stable-baselines3        1.0
termcolor                1.1.0
tifffile                 2022.10.10
torch                    1.7.1
torchvision              0.8.2
tqdm                     4.64.1
typing_extensions        4.4.0
urllib3                  1.26.13
viskit                   0.1     
Werkzeug                 2.2.2
wheel                    0.37.1
zipp                     3.11.0

    Any help would be much appreciated! Kind regards

    opened by daniel-v-e 2
Owner
Murtaza Dalal
Passionate about Machine Learning, Computer Vision, Robotics, and AI. Interested in seamlessly integrating software and hardware into into intelligent systems.
Murtaza Dalal
GitHub https://mihdalal.github.io/raps/
Code for our CVPR 2021 Paper "Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes".

Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes (CVPR 2021) Project page | Paper | Colab | Colab for Drawing App Rethinking Style

CompVis Heidelberg 153 Jan 4, 2023
Doosan robotic arm, simulation, control, visualization in Gazebo and ROS2 for Reinforcement Learning.

Robotic Arm Simulation in ROS2 and Gazebo General Overview This repository includes: First, how to simulate a 6DoF Robotic Arm from scratch using GAZE

David Valencia 12 Jan 2, 2023
HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

HashNeRF-pytorch Instant-NGP recently introduced a Multi-resolution Hash Encodin

Yash Sanjay Bhalgat 616 Jan 6, 2023
Source code for NAACL 2021 paper "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference"

TR-BERT Source code and dataset for "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference". The code is based on huggaface's transformers.

THUNLP 37 Oct 30, 2022
Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Learning-Action-Completeness-from-Points Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal A

Pilhyeon Lee 67 Jan 3, 2023
A PyTorch Library for Accelerating 3D Deep Learning Research

Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research Overview NVIDIA Kaolin library provides a PyTorch API for working with a variety

NVIDIA GameWorks 3.5k Jan 7, 2023
PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short-Term Transformer for Online Action Detection".

Long Short-Term Transformer for Online Action Detection Introduction This is a PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short

null 77 Dec 16, 2022
source code for https://arxiv.org/abs/2005.11248 "Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics"

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics This work will be published in Nature Biomedical

International Business Machines 71 Nov 15, 2022
The code for the NSDI'21 paper "BMC: Accelerating Memcached using Safe In-kernel Caching and Pre-stack Processing".

BMC The code for the NSDI'21 paper "BMC: Accelerating Memcached using Safe In-kernel Caching and Pre-stack Processing". BibTex entry available here. B

Orange 383 Dec 16, 2022
Sky Computing: Accelerating Geo-distributed Computing in Federated Learning

Sky Computing Introduction Sky Computing is a load-balanced framework for federated learning model parallelism. It adaptively allocate model layers to

HPC-AI Tech 72 Dec 27, 2022
Orthogonal Over-Parameterized Training

The inductive bias of a neural network is largely determined by the architecture and the training algorithm. To achieve good generalization, how to effectively train a neural network is of great importance. We propose a novel orthogonal over-parameterized training (OPT) framework that can provably minimize the hyperspherical energy which characterizes the diversity of neurons on a hypersphere. See our previous work -- MHE for an in-depth introduction.

Weiyang Liu 11 Apr 18, 2022
Parameterized Explainer for Graph Neural Network

PGExplainer This is a Tensorflow implementation of the paper: Parameterized Explainer for Graph Neural Network https://arxiv.org/abs/2011.04573 NeurIP

Dongsheng Luo 89 Dec 12, 2022
Optimized primitives for collective multi-GPU communication

NCCL Optimized primitives for inter-GPU communication. Introduction NCCL (pronounced "Nickel") is a stand-alone library of standard communication rout

NVIDIA Corporation 2k Jan 9, 2023
Implementation of parameterized soft-exponential activation function.

Soft-Exponential-Activation-Function: Implementation of parameterized soft-exponential activation function. In this implementation, the parameters are

Shuvrajeet Das 1 Feb 23, 2022
Nb workflows - A workflow platform which allows you to run parameterized notebooks programmatically

NB Workflows Description If SQL is a lingua franca for querying data, Jupyter sh

Xavier Petit 6 Aug 18, 2022
Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked.

actions-includes Allows including an action inside another action (by preprocessing the Yaml file). Instead of using uses or run in your action step,

Tim Ansell 70 Nov 4, 2022
Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

V-Sense 171 Dec 26, 2022
Human Action Controller - A human action controller running on different platforms.

Human Action Controller (HAC) Goal A human action controller running on different platforms. Fun Easy-to-use Accurate Anywhere Fun Examples Mouse Cont

null 27 Jul 20, 2022
The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Action Transformer A Self-Attention Model for Short-Time Human Action Recognition This repository contains the official TensorFlow implementation of t

PIC4SeRCentre 20 Jan 3, 2023