I tried Start_Here.ipynb and Swap_Model.ipynb

Same error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-d6266773d3a4> in <module>()
     12 settings = pixray.apply_settings()
     13 pixray.do_init(settings)
---> 14 pixray.do_run(settings)

11 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
   1129                 return modules[name]
   1130         raise AttributeError("'{}' object has no attribute '{}'".format(
-> 1131             type(self).__name__, name))
   1132 
   1133     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'MyRandomPerspective' object has no attribute 'resample'

Any pointers on getting it to build on Cog/Docker?

% cog run python pixray.py --drawer=pixel --prompt=sunrise --output myfile.png

Building Docker image from environment in cog.yaml...
[+] Building 103.3s (15/17)                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                               0.0s
 => => transferring dockerfile: 1.57kB                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                    0.0s
 => resolve image config for docker.io/docker/dockerfile:1.2                                                                                       2.3s
 => [auth] docker/dockerfile:pull token for registry-1.docker.io                                                                                   0.0s
 => CACHED docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc                  0.0s
 => [internal] load metadata for docker.io/nvidia/cuda:10.2-cudnn8-devel-ubuntu18.04                                                               0.6s
 => [auth] nvidia/cuda:pull token for registry-1.docker.io                                                                                         0.0s
 => [stage-0 1/9] FROM docker.io/nvidia/cuda:10.2-cudnn8-devel-ubuntu18.04@sha256:9843db9f92253080bcaad66e44cf8f5e9fbfef3291e10f854d5e226699cca5d  0.0s
 => [internal] load build context                                                                                                                  0.0s
 => => transferring context: 16.35kB                                                                                                               0.0s
 => CACHED [stage-0 2/9] RUN --mount=type=cache,target=/var/cache/apt apt-get update -qq && apt-get install -qqy --no-install-recommends  make  b  0.0s
 => CACHED [stage-0 3/9] RUN curl https://pyenv.run | bash &&  git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plu  0.0s
 => CACHED [stage-0 4/9] COPY .cog/tmp/build4193742389/cog-0.0.1.dev-py3-none-any.whl /tmp/cog-0.0.1.dev-py3-none-any.whl                          0.0s
 => CACHED [stage-0 5/9] RUN --mount=type=cache,target=/root/.cache/pip pip install /tmp/cog-0.0.1.dev-py3-none-any.whl                            0.0s
 => CACHED [stage-0 6/9] COPY requirements.txt /tmp/requirements.txt                                                                               0.0s
 => ERROR [stage-0 7/9] RUN --mount=type=cache,target=/root/.cache/pip pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt           99.7s
------                                                                                                                                                  
 > [stage-0 7/9] RUN --mount=type=cache,target=/root/.cache/pip pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt:                       
#15 1.899 Looking in links: https://download.pytorch.org/whl/torch_stable.html                                                                          
#15 1.909 Collecting git+https://github.com/bfirsh/taming-transformers.git@7a6e64ee (from -r /tmp/requirements.txt (line 24))                           
#15 1.911   Cloning https://github.com/bfirsh/taming-transformers.git (to revision 7a6e64ee) to /tmp/pip-req-build-gifi16k8
#15 1.911   Running command git clone -q https://github.com/bfirsh/taming-transformers.git /tmp/pip-req-build-gifi16k8
#15 19.36   WARNING: Did not find branch or tag '7a6e64ee', assuming revision or ref.
#15 19.37   Running command git checkout -q 7a6e64ee
#15 20.27 Collecting git+https://github.com/openai/CLIP@573315e (from -r /tmp/requirements.txt (line 25))
#15 20.27   Cloning https://github.com/openai/CLIP (to revision 573315e) to /tmp/pip-req-build-r445l8i3
#15 20.28   Running command git clone -q https://github.com/openai/CLIP /tmp/pip-req-build-r445l8i3
#15 21.91   WARNING: Did not find branch or tag '573315e', assuming revision or ref.
#15 21.91   Running command git checkout -q 573315e
#15 22.28 Collecting git+https://github.com/pvigier/perlin-numpy@6f077f8 (from -r /tmp/requirements.txt (line 26))
#15 22.29   Cloning https://github.com/pvigier/perlin-numpy (to revision 6f077f8) to /tmp/pip-req-build-c575k4q6
#15 22.29   Running command git clone -q https://github.com/pvigier/perlin-numpy /tmp/pip-req-build-c575k4q6
#15 23.62   WARNING: Did not find branch or tag '6f077f8', assuming revision or ref.
#15 23.62   Running command git checkout -q 6f077f8
#15 23.97 Collecting git+https://github.com/fbcotter/pytorch_wavelets (from -r /tmp/requirements.txt (line 42))
#15 23.98   Cloning https://github.com/fbcotter/pytorch_wavelets to /tmp/pip-req-build-wgmygv_3
#15 23.98   Running command git clone -q https://github.com/fbcotter/pytorch_wavelets /tmp/pip-req-build-wgmygv_3
#15 25.98 Collecting git+https://github.com/pixray/aphantasia@d7d43a8 (from -r /tmp/requirements.txt (line 48))
#15 25.99   Cloning https://github.com/pixray/aphantasia (to revision d7d43a8) to /tmp/pip-req-build-4ck41u12
#15 25.99   Running command git clone -q https://github.com/pixray/aphantasia /tmp/pip-req-build-4ck41u12
#15 30.97   WARNING: Did not find branch or tag 'd7d43a8', assuming revision or ref.
#15 30.98   Running command git checkout -q d7d43a8
#15 32.98 Collecting torch==1.9.0+cu102
#15 33.00   Downloading https://download.pytorch.org/whl/cu102/torch-1.9.0%2Bcu102-cp38-cp38-linux_x86_64.whl (831.4 MB)
#15 97.89 /root/.pyenv/pyenv.d/exec/pip-rehash/pip: line 20:    71 Killed                  "$PYENV_COMMAND_PATH" "$@"
------
executor failed running [/bin/sh -c pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt]: exit code: 137
ⅹ Failed to build Docker image: exit status 1

This introduces transparency mask optimisation inspired by https://distill.pub/2018/differentiable-parameterizations/#section-rgba.

It adds a boolean setting called "transparency".

Hello, great project you have here. cog run python pixray.py --drawer=pixel --prompt=prompt --output output.png runs successfully on our Fedora 34 machine, however, it always starts with Using device: cpu which is incredibly slow. We have an Nvidia RTX 2070 with the latest official drivers. Other VQGAN projects run fine on this rig, though I must confess, this is our first time using a dockerized environment with cog, so maybe we're doing something wrong? Anyways, if you could point us to the right place where we can force GPU usage that would be great. Thank you.

Neither I nor my friend have been able to run notebooks we had been running successfully for a while. The Pixel art notebook and pixray palette.

Getting the following error:

I was using a colab notebook where I set the learning_rate. But some time in the last month or so the learning rate I had before is now much too small. How has learning_rate changed?

I get a KeyError for any of the drawer types I select in the PixelDrawer notebook.

  KeyError                                  Traceback (most recent call last)
  <ipython-input-2-d0093cd52676> in <module>()
       43 # pixray.add_settings(iterations=500, display_every=50)
       44 
  ---> 45 settings = pixray.apply_settings()
       46 pixray.do_init(settings)
       47 pixray.do_run(settings)
  
  /content/pixray/pixray.py in apply_settings()
     1569 
     1570     vq_parser = setup_parser(vq_parser)
  -> 1571     class_table[settings_core.drawer].add_settings(vq_parser)
     1572 
     1573     if len(global_pixray_settings) > 0:
  
  KeyError: 'pixel'

When a prompt contains a colon (i.e. "Star Wars: Return of the Sith"), it will currently crash generation. This is a small change to prompt parsing to gracefully handle this case. Before:

>>> parse_prompt("hello world:2")
('hello world', 2.0, -inf)
>>> parse_prompt("hello world:2:15")
('hello world', 2.0, 15.0)
>>> parse_prompt("hello world: my name is Mitchell.")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Volumes/Repos/pixray/pixray.py", line 254, in parse_prompt
    return vals[0], float(vals[1]), float(vals[2])
ValueError: could not convert string to float: ' my name is Mitchell.'

After:

>> parse_prompt("hello world")
('hello world', 1, -inf)
>>> parse_prompt("hello world:2")
('hello world', 2.0, -inf)
>>> parse_prompt("hello world:2:15")
('hello world', 2.0, 15.0)
>>> parse_prompt("hello world: my name is Mitchell")
Warning, failed to parse prompt weights, assuming prompt does not contain weights.
('hello world: my name is Mitchell', 1, -inf)

originally the init_weigt returns a cur_loss that is a dimension 0 tensor (like this [#]), but we need just a float tensor (like this #), so changed that. additionally, the z_orig does not actually reflect the init_image, rather the added noise init image, so i changed that statement.

tested with vqgan, current doesn't work with pixel drawer

It is very handy that pixray has such smooth support for both "square" and "widescreen" aspects.

I was wondering if you'd be open to also adding a "portrait" aspect, which would be effective for, well, portraits, but also various vertical compositions (e.g. individual waterfalls, skyscrapers, trees, and so on)

Hello @dribnet , i tried running this on colab, but i always get error "CUDA out of memory". i tried to set quality to draft but still got the same result.

any solution for this ?

thank you

Hey @dribnet 👋🏼

I just forked this repo and started to work on a few changes before noticing the README message about it being archived: https://github.com/dribnet/pixray/commit/f2a29ac5c355cb0c82a665a5edbd94751694e1a4

GitHub has a feature for archiving repos which puts a noticeable banner at the top of the page and moves everything like issues and PRs into a read-only state. See https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories

This could be a useful switch to flip to make sure people don't waste time opening or commenting on issues or PRs in the wrong place.

Apologies for rubbish formatting, but Github's code insert is broken.

Using Pixray Pixeldraw, with the settings as they come, aside from: aspect: square drawer: vqgan

` /content/pixray/vqgan.py in load_model(self, settings, device) 156 self.e_dim = model.quantize.e_dim 157 self.n_toks = model.quantize.n_e --> 158 self.z_min = model.quantize.embedding.weight.min(dim=0).values[None, :, None, None] 159 self.z_max = model.quantize.embedding.weight.max(dim=0).values[None, :, None, None] 160

RuntimeError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 15.90 GiB total capacity; 14.78 GiB already allocated; 15.75 MiB free; 14.88 GiB reserved in total by PyTorch) `

Using GPU on Colab Pro (again github code insert doesn't work): +-----------------------------------------------------------------------------+ | NVIDIA-SMI 495.44 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 | | N/A 42C P0 32W / 250W | 16265MiB / 16280MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

I really enjoy the wallpaper filter but if i want to save an image while it is running i am stuck with those ever-changing dashed lines that ruin the image. Is there a way that this could be done in the background without having the dashed black lines?

Hi, I could successfully run pixray on colab (pro version). However, when I tried to change the drawer style to "clipdraw" I ran into some error message:

/usr/local/lib/python3.7/dist-packages/diffvg-0.0.1-py3.7-linux-x86_64.egg/pydiffvg/render_pytorch.py in backward(ctx, grad_img)
    707                       use_prefiltering,
    708                       diffvg.float_ptr(eval_positions.data_ptr()),
--> 709                       eval_positions.shape[0])
    710         time_elapsed = time.time() - start
    711         global print_timing

RuntimeError: radix_sort: failed on 1st step: cudaErrorInvalidDevice: invalid device ordinal

It may be related to the memory, however, I was using a P100 GPU:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44       Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   37C    P0    37W / 250W |      0MiB / 16280MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I used the following code to install pytorch:

%%capture
# this patch applied to fix torchtext dependency 11 Nov 2021
!pip install torch==1.9.0+cu111 torchtext==0.10.0 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch/ -f https://download.pytorch.org/whl/torchvision/

!git clone https://github.com/openai/CLIP
# !pip install taming-transformers
!git clone https://github.com/CompVis/taming-transformers.git
!rm -Rf pixray
!git clone https://github.com/dribnet/pixray
!pip install ftfy regex tqdm omegaconf pytorch-lightning
!pip install kornia==0.6.1
!pip install imageio-ffmpeg   
!pip install einops
!pip install torch-optimizer
!pip install easydict
!pip install braceexpand
!pip install git+https://github.com/pvigier/perlin-numpy
!mkdir -p steps
!mkdir -p models

!pip install svgwrite
!pip install svgpathtools
!pip install cssutils
!pip install numba
!pip install torch-tools
!pip install visdom
!pip install gradio==2.3.7

!git clone https://github.com/BachiLi/diffvg
%cd diffvg
# !ls
!git submodule update --init --recursive
!python setup.py install
%cd ..

!pip install aphantasia

import sys
sys.path.append("pixray")

I'm trying to run the Docker version of https://replicate.com/dribnet/pixray-api using an Nvidia 3090.

docker run -d -p 5000:5000 --gpus=all r8.im/dribnet/pixray-api@sha256:a249606da3a0c7f32eed4741f1e6f1792470a39a5825fc8814272cceea30ad32
curl http://localhost:5000/predict -X POST \
  -F settings=...

I'm getting this error

/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/cuda/__init__.py:106: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

I suspect the problem is CUDA version (CUDA11.1 ) ( see https://discuss.pytorch.org/t/geforce-rtx-3090-with-cuda-capability-sm-86-is-not-compatible-with-the-current-pytorch-installation/123499/8 )

Is it possible to upgrade CUDA version ing cogs.yaml?

Love this work! However, as I've been toying around with pixray the last day or so, and have had quite a hard time installing everything from scratch... all the git submodule business + local setup.py installs + (😱 gasp) appending to sys path to make things work feels really sketchy!

Is a PyPi package on your roadmap? Something where I could go pip install pixray (or pip install pixray[<some-extra>] if you wanted to separate out the extra demo dependencies) would be super nice and would make this work a lot more accessible to folks!