Hi, this is really an impressive work! Two question here.

1. I would like ask is the overall process of the text-guided image editing is using only pre-trained model without any extra training or fine-tuning?
2. If it does not required any further fine-tuning or training, what is the purpose of having diffusion guided loss (which combine loss from CLIP model and background preservation loss)?

Thanks in advance for your clarification!

in image_editor.py:

self.model.load_state_dict( torch.load( "checkpoints/256x256_diffusion_uncond.pt" if self.args.model_output_size == 256 else "checkpoints/512x512_diffusion.pt", map_location="cpu", ) )

mentioned '512x512_diffusion', is it an conditional or unconditional model? (can you share me its download link?) It is natural that your method is built on a pretrained uncontitional model. If '512x512_diffusion' is a conditioonal model, what is the condition for face data for example? can you help me figure out this?

Hi! I wonder if a loss such as MSE or LPIPS is used between the user-provided scribbles and the scribbled regions of $\widehat{x}_0$ , in addition to the CLIP loss. I am curious how the shapes and colors stay consistent when only text with no specific description, e.g., "blanket" in Fig 9, is given.

Thanks for opening your code. I really appreciated that.

I tried to run your code in the Google Colab with GPU Runtime.

I got an error. And I couldn't find any solution despite of googling...

The error message is as follows : AttributeError: 'PosixPath' object has no attribute 'with_stem'

I think it's kinda related to "pathlib" module. maybe it's due to the fact that pathlib doesn't work well with latest python 3.x version which I'm using.

I hope this error will be solved soon QQ

-----detail description-------

I run the terminal argument like this as follows : python main.py -p "rock" -i "input_example/img.png" --mask "input_example/mask.png" --output_path "output" --batch_size 1

And, I got this whole bunch of error messages as follow : Using device: cuda:0 tcmalloc: large alloc 2209964032 bytes == 0x89ebe000 @ 0x7fbdefa2cb6b 0x7fbdefa4c379 0x7fbd30b1026e 0x7fbd30b119e2 0x7fbd334aeee1 0x7fbdd598e236 0x7fbdd541ef98 0x593784 0x594731 0x548cc1 0x51566f 0x549e0e 0x593fce 0x5118f8 0x549e0e 0x4bcb19 0x59582d 0x595b69 0x62026d 0x55de15 0x59af67 0x515655 0x549e0e 0x4bca8a 0x5134a6 0x549576 0x593fce 0x548ae9 0x5127f1 0x4bc98a 0x532b86 Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off] Loading model from: /usr/local/lib/python3.7/dist-packages/lpips/weights/v0.1/vgg.pth Start iterations 0 0% 0/75 [00:00<?, ?it/s]clip_loss - 867.99 range_loss - 0.00

Traceback (most recent call last): File "main.py", line 8, in <module> image_editor.edit_image_by_prompt()

File "/content/drive/MyDrive/ws/blended-diffusion/optimization/image_editor.py", line 266, in edit_image_by_prompt visualization_path = visualization_path.with_stem(

AttributeError: 'PosixPath' object has no attribute 'with_stem' 0% 0/75 [00:01<?, ?it/s]

The codes below are the ones that error comes in. Those are from blended-diffusion/optimization/image_editor.py python file.

line 1 - from pathlib import Path ... line 261 - for b in range(self.args.batch_size): line 262 - pred_image = sample["pred_xstart"][b] line 263 - visualization_path = Path( line 264 - os.path.join(self.args.output_path, self.args.output_file) line 265 - ) line 266 - visualization_path = visualization_path.with_stem(

Hi, I would like to ask what is the purpose of having skip_timesteps in the code? I can't seems to find any related information on this from the paper.