Hi,

Nice paper and code. They're very clear and easy to understand. I have two questions about the code implementation:

1. Although I've read articles about the background of the Diffusion model, I'm still new to it. I understand the loss function of comparing epsilons and using p process (the U-Net model) to estimate noise, but I'm confused why you set self.predict_epsilon == False in the locomotion example. That's amazing. What's the loss function and logic behind it? Could you briefly explain or refer me to somewhere to read please?

2. I think we can use the same model for path planning in Maze 2D, but I'm not sure about the code implementation. Please correct me if I'm wrong, a. At initialization, the points are in order which represent the visiting order, and their 2D positions are random. b. The constraints are (1) the first/last points should be at the start/end position, (2) the path between consecutive should not hit/cross the walls (I don't know how to implement constraint yet). c. Both training and testing stage, we use the same function apply_condition to apply the constraints.

Thanks for your time. Look forward to your response.

Best

Hi,

Thanks for sharing the nice code! There was a minor issue when I trained the model with the script python scripts/train.py --dataset maze2d-umaze-v1 --horizon 512 --n_diffusion_steps 200:

File ".../diffuser/datasets/buffer.py", line 75, in add_path
    self._dict[key][self._count, :path_length] = array
IndexError: index 10000 is out of bounds for axis 0 with size 10000

and I believe the episode number of the maze2d dataset is more than 10000, which caused this error. So I set max_n_episodes=20000 at the line: https://github.com/jannerm/diffuser/blob/main/diffuser/datasets/sequence.py#L18. And it really fixed this issue. Maybe you can update the corresponding code if I'm correct?

BTW, looking forward to seeing your testing/evaluating code :)

Hi,

I'm really interested in your work. It's awesome. Do you have any plans to release training/testing codes and configs for Maze the Block stacking datasets? It would be a great help to understand and follow this paper better.

Thanks.

Hi,

I tried to create a conda virtual environment but failed. So I turn to create a docker, but it fails at the same place, in creating the virtual environment.

#19 484.3   subprocess.CalledProcessError: Command '['/opt/conda/envs/diffuser/bin/python', 'dm_control/autowrap/autowrap.py', '--header_paths=/root/.mujoco/mujoco200_linux/include/mjdata.h /root/.mujoco/mujoco200_linux/include/mjmodel.h /root/.mujoco/mujoco200_linux/include/mjrender.h /root/.mujoco/mujoco200_linux/include/mjui.h /root/.mujoco/mujoco200_linux/include/mjvisualize.h /root/.mujoco/mujoco200_linux/include/mjxmacro.h /root/.mujoco/mujoco200_linux/include/mujoco.h', '--output_dir=/tmp/pip-install-alxuyfnp/dm-control_837352932d0a4468a1a9f22aec04744b/build/lib/dm_control/mujoco/wrapper/mjbindings']' returned non-zero exit status 1.                                                                        
#19 484.3   ----------------------------------------                                                                    
#19 484.3   ERROR: Failed building wheel for dm-control                                                                 
#19 484.3 ERROR: Could not build wheels for mujoco-py which use PEP 517 and cannot be installed directly 

I guess the problem is related to dm-control. Do you have any suggestion to fix it? Thanks.

Hi there, I checked out the plan_maze2d branch and tried to run "conda env create -f environment.yml" and bump into the following errors. Any guidance on how to solve this issue? Thank you very much!

Collecting dm_control@ git+git://github.com/deepmind/dm_control@41d0c7383153f9ca6c12f8e865ef5e73a98759bd#egg=dm_control Cloning git://github.com/deepmind/dm_control (to revision 41d0c7383153f9ca6c12f8e865ef5e73a98759bd) to /tmp/pip-install-ncjf6ze3/dm-control_fcf322da5da84fb9a7196336199dff53

Pip subprocess error: Running command git clone --filter=blob:none --quiet https://github.com/JannerM/d4rl.git /tmp/pip-req-build-tmbo87ge Running command git rev-parse -q --verify 'sha^13fc08c198d17ac627f528aedc70843e8f86bb71' Running command git fetch -q https://github.com/JannerM/d4rl.git 13fc08c198d17ac627f528aedc70843e8f86bb71 Running command git checkout -q 13fc08c198d17ac627f528aedc70843e8f86bb71 WARNING: Missing build requirements in pyproject.toml for mujoco-py==2.0.2.13 from https://files.pythonhosted.org/packages/2f/48/b108057c1a23c8da9f4cdc7a7c46ab7cec49c3563c0706d50f2527de6ba0/mujoco-py-2.0.2.13.tar.gz (from -r /home/external/diffuser/condaenv.75wnjbo_.requirements.txt (line 4)). WARNING: The project does not specify a build backend, and pip cannot fall back to setuptools without 'wheel'. Running command git clone --filter=blob:none --quiet git://github.com/deepmind/dm_control /tmp/pip-install-ncjf6ze3/dm-control_fcf322da5da84fb9a7196336199dff53 fatal: unable to connect to github.com: github.com[0: 192.30.255.113]: errno=Connection timed out

error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet git://github.com/deepmind/dm_control /tmp/pip-install-ncjf6ze3/dm-control_fcf322da5da84fb9a7196336199dff53 did not run successfully. │ exit code: 128 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet git://github.com/deepmind/dm_control /tmp/pip-install-ncjf6ze3/dm-control_fcf322da5da84fb9a7196336199dff53 did not run successfully. │ exit code: 128 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

failed

CondaEnvException: Pip failed

Hi. I found the code's defaults hyperparameters is inconsistent with the paper's such as diffusion steps N=100 in paper but N=20 in code and guide scale, gamma, train step. I wonder how much this will affect performance and if the code's hyperparameters is enough to obtain results reported in paper. Thanks if anyone can give me some helps.

Hi,

Thanks for sharing your code and the implementation details. I was trying to reproduce the stacking experiments but I wasn't able to find the exact training scripts or code for this. I'm terribly sorry if I missed this in the repo. Would it be possible to release this env/code for these experiments. Many thanks in advance.

When I try the following command—— python scripts/train.py --dataset halfcheetah-medium-expert-v2

There is an error occurring—— gcc: error: unrecognized command line option ‘-R’ Traceback (most recent call last): File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 204, in link self.spawn(linker + ld_args) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 910, in spawn spawn(cmd, dry_run=self.dry_run, **kwargs) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 87, in spawn raise DistutilsExecError( distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "scripts/train.py", line 1, in <module> import diffuser.utils as utils File "/home/charlie/PycharmProjects/diffuser/diffuser/utils/__init__.py", line 6, in <module> from .rendering import * File "/home/charlie/PycharmProjects/diffuser/diffuser/utils/rendering.py", line 8, in <module> import mujoco_py as mjc File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/__init__.py", line 3, in <module> from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 510, in <module> cymj = load_cython_ext(mujoco_path) File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 105, in load_cython_ext cext_so_path = builder.build() File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 221, in build built_so_file_path = self._build_impl() File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 291, in _build_impl so_file_path = super()._build_impl() File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 244, in _build_impl dist.run_commands() File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands self.run_command(cmd) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 341, in run self.build_extensions() File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/mujoco_py/builder.py", line 144, in build_extensions build_ext.build_extensions(self) File "/home/charlie/anaconda3/envs/diffuser/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 450, in build_extensions self._build_extensions_serial() File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 475, in _build_extensions_serial self.build_extension(ext) File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 552, in build_extension self.compiler.link_shared_object( File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 713, in link_shared_object self.link(CCompiler.SHARED_OBJECT, objects, File "/home/charlie/.local/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 206, in link raise LinkError(msg) distutils.errors.LinkError: command '/usr/bin/gcc' failed with exit code 1

This seems to be an error about GCC. Could you tell me the versions of gcc and g++ you used or Do you have other better solutions?

Thanks for any help!

Hello, Thanks for the open sourcing codes! But I have one question about the implementation of training values. Should the input of this line be x_start instead of x_noisy? Because the target is accumulated return calculated from the expert trajectory of offline data and not the true return of denoising trajectories during the denoising or noisy trajecotories, which means the gradient guide from target is not matching to x_noisy. I don't know whether I am right about this, please take a look and offer me some hints. https://github.com/jannerm/diffuser/blob/cbcd3fb6f2a95538de19604f0621903bb160d0a1/diffuser/models/diffusion.py#L243

Hi and thank you for sharing the code of this very interesting work!

Do you intend to share the maze2D weights? In the google folder with the trained models there is no trained model for the maze2d. Am I missing something?

Running the plan_maze2d.py I get the following error: FileNotFoundError: [Errno 2] No such file or directory: 'logs/maze2d-large-v1/diffusion/H384_T256/model_config.pkl'

My folder structure after running the script:

Any help would be great!

I ran the training steps with Docker

docker build -f azure/Dockerfile . -t diffuser
docker run -it --rm --gpus all \
    --mount type=bind,source=$PWD,target=/home/code \
    --mount type=bind,source=$HOME/.d4rl,target=/root/.d4rl \
    diffuser \
    bash -c \
    "export PYTHONPATH=$PYTHONPATH:/home/code && \
    python /home/code/scripts/train.py --dataset hopper-medium-expert-v2 --logbase logs/docker"

However after 200k+ iterations it did not appear to increase the episode length. Could you provide more details on how you trained your agent and how I can reproduce it?

Hi there, first of all, thanks for making this work available. I wanted to go through the Colab Notebook provided at the header of the readme file. In the second block of the Setup section, there are some errors that block us from running the following blocks. Changing it from

%cd /content

!git clone https://github.com/jannerm/diffuser.git
!git checkout 06b8e6a042e6a3312d50ed8048cba14afeab3085
!curl -L -o diffuser-pretrained.tar https://www.dropbox.com/s/1x73p8x66mj7dvk/diffuser-pretrained.tar?dl=1
!tar -xf diffuser-pretrained.tar
!mv diffuser-pretrained/logs diffuser/logs
!rm -r diffuser-pretrained*

%cd diffuser
%pip install -f https://download.pytorch.org/whl/torch_stable.html \
                typed-argument-parser \
                scikit-image==0.17.2 \
                scikit-video==1.1.11 \
                gitpython \
                einops \
                pillow \
                free-mujoco-py \
                gym \
                git+https://github.com/JannerM/d4rl.git@0e84a4d29db3ae501043215ce1d91843929f1949 \
                git+https://github.com/aravindr93/mjrl

%pip install -e .

to

%cd /content

!git clone https://github.com/jannerm/diffuser.git
%cd diffuser
!git checkout 06b8e6a042e6a3312d50ed8048cba14afeab3085
%cd /content
!curl -L -o diffuser-pretrained.tar https://www.dropbox.com/s/1x73p8x66mj7dvk/diffuser-pretrained.tar?dl=1
!tar -xf diffuser-pretrained.tar
!mv diffuser-pretrained/logs diffuser/logs
!rm -r diffuser-pretrained*

%cd diffuser
%pip install -f https://download.pytorch.org/whl/torch_stable.html \
                typed-argument-parser \
                scikit-image==0.17.2 \
                scikit-video==1.1.11 \
                gitpython \
                einops \
                pillow \
                free-mujoco-py \
                gym==0.18.0 \
                git+https://github.com/JannerM/d4rl.git@0e84a4d29db3ae501043215ce1d91843929f1949 \
                git+https://github.com/aravindr93/mjrl

%pip install -e .

solves the issue. Maybe you might want to change the corresponding part of ./scripts/diffuser_sample.ipynb as well :)

Best, Kagan

I think when we start docker to train the model, we need to see the log files in the host, so I bind the /logs/docker to a local directory. And because in Dockerfile, the environment path is already set. I remove the corresponding bash command.