I did not see a Discussion section, so asking here in Issues: (1) Has anyone published a Google Colab notebook? and (2) how was the animation created of the vase?

For (1), I can whip one up assuming that the CUDA matches what Colab gives. Otherwise, this needs tweaking

For (2), it looks like you are varying the seed and/or the text prompt and then perhaps creating keyframes which are then interpolated? It is a cool effect. The other would be to explore 3D printing of things like "the cactus shoe"

Hi! Do you know, maybe, how to run the code on GPU with CUDA 11.3 or 11.2? AFAIU, one would need pytorch 1.10 for this, but then kaolin asks for pytorch <= 1.9

Running the demo scripts on a fresh clone of the repo throws an error:

(text2mesh) cpacker@...:text2mesh$ ./demo/run_alien_cobble.sh
Traceback (most recent call last):
  File "text2mesh/main.py", line 499, in <module>
    run_branched(args)
  File "text2mesh/main.py", line 34, in run_branched
    clip_model, preprocess = clip.load(args.clipmodel, device, jit=args.jit)
  File "anaconda3/envs/text2mesh/lib/python3.9/site-packages/clip/clip.py", line 124, in load
    raise RuntimeError(f"Model {name} not found; available models = {available_models()}")
RuntimeError: Model VIT-B/32 not found; available models = ['RN50', 'RN101', 'RN50x4', 'RN50x16', 'RN50x64', 'ViT-B/32', 'ViT-B/16', 'ViT-L/14', 'ViT-L/14@336px']

Seems like this is due to a typo here, VIT should be ViT.

Same command post-fix:

(text2mesh) cpacker@...:text2mesh$ ./demo/run_alien_cobble.sh
ModuleList(
  (0): FourierFeatureTransform()
  (1): ProgressiveEncoding()
  (2): Linear(in_features=515, out_features=256, bias=True)
  (3): ReLU()
  (4): Linear(in_features=256, out_features=256, bias=True)
  (5): ReLU()
  (6): Linear(in_features=256, out_features=256, bias=True)
  (7): ReLU()
  (8): Linear(in_features=256, out_features=256, bias=True)
  (9): ReLU()
  (10): Linear(in_features=256, out_features=256, bias=True)
  (11): ReLU()
)
ModuleList(
  (0): Linear(in_features=256, out_features=256, bias=True)
  (1): ReLU()
  (2): Linear(in_features=256, out_features=256, bias=True)
  (3): ReLU()
  (4): Linear(in_features=256, out_features=3, bias=True)
)
ModuleList(
  (0): Linear(in_features=256, out_features=256, bias=True)
  (1): ReLU()
  (2): Linear(in_features=256, out_features=256, bias=True)
  (3): ReLU()
  (4): Linear(in_features=256, out_features=1, bias=True)
)
  0%|                                                                                                                                           | 0/1500 [00:00<?, ?it/s]

Update parser options to take in any CLIP model + with JIT. The render image resolution is updated according to the model specifications. Note: as new CLIP models become public, for the sake of efficiency it will probably be good to just replace the render preprocessing (e.g. setting the render resolution) with the preprocess function given in the CLIP model loading. We can just set the default render resolution to be the current max of the models to prevent too much blurring from upsampling.

Results shown on the ninja with each model with the same options + seed: RN50 RN101 RN50x4 RN50x16 RN50x64 VIT-B/32 VIT-B/16 VIT-L/14 VIT-L/14@336px

Instead of using the pip installation method of yml, install it according to the methods on the clip and kaolin homepages, just make sure that there are clip1.0 and kaolin 0.12.0 in the conda list.

1. Install clip1.0: pip install git+https://github.com/openai/CLIP.git 2. Install kaolin0.12.0: vi ~/.bashrc export CUDA_HOME=/usr/local/cuda source ~/.bashrc conda activate text2mesh git clone --recursive https://github.com/NVIDIAGameWorks/kaolin cd kaolin git checkout v0.12.0 python setup.py develop

Please refer to the following URL clip 1.0 kaolin 0.12.0 For more details please refer to my blog My Blog

Hello,

I have no errors when I run conda env create --file text2mesh.yml and conda activate text2mesh commands.

But when I execute the ./demo/run_candle.sh file, I get this error : Traceback (most recent call last): File "C:\Users\matth\text2mesh\main.py", line 3, in <module> import kaolin.ops.mesh ModuleNotFoundError: No module named 'kaolin'

I have also the same error with the clip module.

I am supposed to be in the env with everything perfectly installed.

Please help me

I removed --prompt from the settings of run_shoe.sh and set values for --no_prompt and --image, and the quality of results were really bad. Details are as follows:

case1

• run_shoe.sh python main.py --run branch --obj_path data/source_meshes/shoe.obj --output_dir results/demo/shoe/texture/brick --no_prompt --image data/target_texture/brick_texture.jpg --sigma 5.0 --clamp tanh --n_normaugs 4 --n_augs 1 --normmincrop 0.1 --normmaxcrop 0.1 --geoloss --colordepth 2 --normdepth 2 --frontview --frontview_std 4 --clipavg view --lr_decay 0.9 --clamp tanh --normclamp tanh --maxcrop 1.0 --save_render --seed 11 --n_iter 1500 --learning_rate 0.0005 --normal_learning_rate 0.0005 --background 1 1 1 --frontview_center 0.5 0.6283

• brick_texture.jpg

• result

case2

• run_shoe.sh python main.py --run branch --obj_path data/source_meshes/shoe.obj --output_dir results/demo/shoe/texture2/cactus --no_prompt --image data/target_texture/cactus_texture.jpg --sigma 5.0 --clamp tanh --n_normaugs 4 --n_augs 1 --normmincrop 0.1 --normmaxcrop 0.1 --geoloss --colordepth 2 --normdepth 2 --frontview --frontview_std 4 --clipavg view --lr_decay 0.9 --clamp tanh --normclamp tanh --maxcrop 1.0 --save_render --seed 11 --n_iter 1500 --learning_rate 0.0005 --normal_learning_rate 0.0005 --background 1 1 1 --frontview_center 0.5 0.6283

• cactus_texture.jpg

• result

Did I set the parameter wrong? Or is there something in main.py that needs to be modified? Or is it a randomness issue in optimization?

When --no_prompt is set to True and --image is set to image path string in main.py, the loss code corresponding to 'local to global' and 'local to displacement' in the paper is not understood. Should I change this part?

Thanks for your excellent work! When I run your demo, I notice that the input points are sent to progressive encoding. If I disable this module, the final result dosen't change a lot. So why do we need this module? Here is the two picture of results( promt: an image of a shoe made of cactus, obj: shoe), the first one is WITH progressive encoding, the second one is WITHOUT progressive encoding.