Text2Art is an AI art generator powered with VQGAN + CLIP and CLIPDrawer models

Overview

thumbnail

Text2Art

AI Powered Text-to-Art Generator
Try It Out »
View Notebook · Read Article · Report Bug · Request Feature

About The Project

Text2Art is an AI art generator powered with VQGAN + CLIP and CLIPDrawer models. You can easily generate all kind of art from drawing, painting, sketch, or even a specific artist style just using a text input. You can also specify the dimensions of the image. The process can take 3-20 mins and the results will be emailed to you.

You can read the write-up on how I built this project in this article

demo

Built With

Getting Started

Open In Colab

You can start with the colab notebook to generate an art manually using code. If you want to deploy a website like text2art, make sure you setup your MailGun and Firebase account and replace the API key accordingly.

License

Distributed under the MIT License. See LICENSE for more information.

Contact

You can drop me a message on my Website.

Comments
  • Runttime errors in Google Colab on requires_grad_ is not supported on ScriptModules

    Runttime errors in Google Colab on requires_grad_ is not supported on ScriptModules

    Hi, thank you for sharing this repo. I wondered if you could help me get it to run in the Colab, because it does not seem run as-is. I know Python but I am a noob in this ML domain.

    Off the bat it was failing on import clipit on a call to torchtext which appeared to be versioning issue. I added !pip install -U torchtextwhich then led to this failure further down the code.

    RuntimeError                              Traceback (most recent call last)
    <ipython-input-2-25d790141602> in <module>()
         12 # Apply these settings and run
         13 settings = clipit.apply_settings()
    ---> 14 clipit.do_init(settings)
         15 clipit.do_run(settings)
    
    1 frames
    
    /usr/local/lib/python3.7/dist-packages/torch/jit/_script.py in fail(self, *args, **kwargs)
        912     def _make_fail(name):
        913         def fail(self, *args, **kwargs):
    --> 914             raise RuntimeError(name + " is not supported on ScriptModules")
        915 
        916         return fail
    
    RuntimeError: requires_grad_ is not supported on ScriptModules
    
    opened by metalaureate 6
  • Google Colab 'MyRandomPerspective' object has no attribute 'resample'

    Google Colab 'MyRandomPerspective' object has no attribute 'resample'

    Hi there,

    When I try to run your Google Colab notebook, I got an error that says:


    AttributeError Traceback (most recent call last) in () 13 settings = clipit.apply_settings() 14 clipit.do_init(settings) ---> 15 clipit.do_run(settings)

    11 frames /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in getattr(self, name) 1129 return modules[name] 1130 raise AttributeError("'{}' object has no attribute '{}'".format( -> 1131 type(self).name, name)) 1132 1133 def setattr(self, name: str, value: Union[Tensor, 'Module']) -> None:

    AttributeError: 'MyRandomPerspective' object has no attribute 'resample'

    Could you please help? Thanks!

    opened by oonacitarhan 4
  • error

    error

    ImportError Traceback (most recent call last) in () ----> 1 import clipit 2 # To reset settings to default 3 clipit.reset_settings() 4 # You can use "|" to separate multiple prompts 5 prompts = "underwater city"

    10 frames /usr/local/lib/python3.7/dist-packages/torch_xla/init.py in () 126 import torch 127 from ._patched_functions import _apply_patches --> 128 import _XLAC 129 130

    ImportError: /usr/local/lib/python3.7/dist-packages/_XLAC.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at11result_typeERKNS_6TensorEN3c106ScalarE


    NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt.

    To view examples of installing some common dependencies, click the "Open Examples" button below.

    opened by buddhix 3
  • Error

    Error

    AttributeError Traceback (most recent call last) in () 1 import clipit 2 # To reset settings to default ----> 3 clipit.reset_settings() 4 # You can use "|" to separate multiple prompts 5 prompts = "death sky rain"

    AttributeError: module 'clipit' has no attribute 'reset_settings'

    Always get this error when using the Colab. I run setup, restart runtime and then execute the next block. It breaks then.

    opened by CEM1204 3
  • Segmentation fault (core dumped)

    Segmentation fault (core dumped)

    First I was getting error like :

    RuntimeError: requires_grad_ is not supported on ScriptModules

    Found one solution at : https://githubhelp.com/mfrashad/text2art/issues/5 Solution says change version of tochtext to torchtext==0.10.0 So installed using : pip install torchtext==0.10.0 It changed torch version to 1.9.0 from 1.10.0

    Now getting another error :

    /home/ubuntu/anaconda3/envs/textToArt/lib/python3.7/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: /home/ubuntu/anaconda3/envs/textToArt/lib/python3.7/site-packages/torchvision/image.so: undefined symbol: _ZNK2at10TensorBase21__dispatch_contiguousEN3c1012MemoryFormatE warn(f"Failed to load image Python extension: {e}") Segmentation fault (core dumped)

    opened by Unmesh28 2
  • clipit fails to import

    clipit fails to import

    Hi, trying to run this from Google Colab, returns this error:

    ImportError                               Traceback (most recent call last)
    <ipython-input-17-25d790141602> in <module>()
    ----> 1 import clipit
          2 # To reset settings to default
          3 clipit.reset_settings()
          4 # You can use "|" to separate multiple prompts
          5 prompts = "underwater city"
    
    /content/clipit/clipit.py in <module>()
         22 #torch.use_deterministic_algorithms(True)               # NR: grid_sampler_2d_backward_cuda does not have a deterministic implementation
         23 
    ---> 24 from torch_optimizer import DiffGrad, AdamP, RAdam
         25 from perlin_numpy import generate_fractal_noise_2d
         26 
    
    ImportError: cannot import name 'RAdam' from 'torch_optimizer' (/usr/local/lib/python3.7/dist-packages/torch_optimizer/__init__.py)
    
    opened by szymekdr 2
  • I am getting the same error 😢😭

    I am getting the same error 😢😭

    error

    please help here is the error 👇:

    `AttributeError
    Traceback (most recent call last) /tmp/ipykernel_32/3491049020.py in 1 import clipit 2 # To reset settings to default ----> 3 clipit.reset_settings() 4 # You can use "|" to separate multiple prompts 5 prompts = "underwater city"

    AttributeError: module 'clipit' has no attribute 'reset_settings'

    `

    opened by shreesha345 1
  • clipit import error

    clipit import error

    ModuleNotFoundError Traceback (most recent call last) in 1 import sys 2 sys.path.append("clipit") ----> 3 import clipit 4 5 # To reset settings to default

    /content/clipit/clipit.py in 25 from perlin_numpy import generate_fractal_noise_2d 26 ---> 27 from CLIP import clip 28 import kornia 29 import kornia.augmentation as K

    ModuleNotFoundError: No module named 'CLIP'


    NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt.

    To view examples of installing some common dependencies, click the "Open Examples" button below.

    opened by shakil033 1
  • error error error full of this

    error error error full of this

    ```Working` with z of shape (1, 256, 16, 16) = 65536 dimensions. loaded pretrained LPIPS loss from taming/modules/autoencoder/lpips/vgg.pth VQLPIPSWithDiscriminator running with hinge loss. Restored from models/vqgan_imagenet_f16_16384.ckpt Using device: cuda:0 Optimising using: Adam Using text prompts: ['underwater city'] Using seed: 1698681138380486500 0/? [00:00<?, ?it/s] Oops: runtime error: solve: MAGMA library not found in compilation. Please rebuild with MAGMA. Try reducing --num-cuts to save memory

    RuntimeError Traceback (most recent call last) /tmp/ipykernel_58/2225298613.py in 21 settings = clipit.apply_settings() 22 clipit.do_init(settings) ---> 23 clipit.do_run(settings)

    /kaggle/working/clipit/clipit.py in do_run(args) 997 print("Oops: runtime error: ", e) 998 print("Try reducing --num-cuts to save memory") --> 999 raise e 1000 except KeyboardInterrupt: 1001 pass

    /kaggle/working/clipit/clipit.py in do_run(args) 989 while True: 990 try: --> 991 train(args, cur_iteration) 992 if cur_iteration == args.iterations: 993 break

    /kaggle/working/clipit/clipit.py in train(args, cur_it) 902 903 for i in range(args.batches): --> 904 lossAll = ascend_txt(args) 905 906 if i == 0 and cur_it % args.save_every == 0:

    /kaggle/working/clipit/clipit.py in ascend_txt(args) 723 for cutoutSize in cutoutsTable: 724 make_cutouts = cutoutsTable[cutoutSize] --> 725 cur_cutouts[cutoutSize] = make_cutouts(out) 726 727 if args.spot_prompts:

    /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks 1050 or _global_forward_hooks or _global_forward_pre_hooks): -> 1051 return forward_call(*input, **kwargs) 1052 # Do not call functions when jit is used 1053 full_backward_hooks, non_full_backward_hooks = [], []

    /kaggle/working/clipit/clipit.py in forward(self, input, spot) 352 # TF.to_pil_image(batch[j_wide].cpu()).save(f"cached_im_{cur_iteration:02d}{j_wide:02d}{spot}.png") 353 else: --> 354 batch1, transforms1 = self.augs_zoom(torch.cat(cutouts[:self.cutn_zoom], dim=0)) 355 batch2, transforms2 = self.augs_wide(torch.cat(cutouts[self.cutn_zoom:], dim=0)) 356 # print(batch1.shape, batch2.shape)

    /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks 1050 or _global_forward_hooks or _global_forward_pre_hooks): -> 1051 return forward_call(*input, **kwargs) 1052 # Do not call functions when jit is used 1053 full_backward_hooks, non_full_backward_hooks = [], []

    /opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input) 137 def forward(self, input): 138 for module in self: --> 139 input = module(input) 140 return input 141

    /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks 1050 or _global_forward_hooks or _global_forward_pre_hooks): -> 1051 return forward_call(*input, **kwargs) 1052 # Do not call functions when jit is used 1053 full_backward_hooks, non_full_backward_hooks = [], []

    /opt/conda/lib/python3.7/site-packages/kornia/augmentation/augmentation.py in forward(self, input, params, return_transform) 1141 input_pad = self.compute_padding(input_temp.shape) 1142 _input = self.precrop_padding(input_temp, input_pad) # type: ignore -> 1143 out = super().forward(_input, params, return_transform) 1144 1145 # Update the actual input size for inverse

    /opt/conda/lib/python3.7/site-packages/kornia/augmentation/base.py in forward(self, input, params, return_transform) 243 244 self._params = params --> 245 output = self.apply_func(in_tensor, in_transform, self._params, return_transform) 246 return _transform_output_shape(output, ori_shape) if self.keepdim else output 247

    /opt/conda/lib/python3.7/site-packages/kornia/augmentation/base.py in apply_func(self, in_tensor, in_transform, params, return_transform) 202 # if all data needs to be augmented 203 elif torch.sum(to_apply) == len(to_apply): --> 204 trans_matrix = self.compute_transformation(in_tensor, params) 205 output = self.apply_transform(in_tensor, params, trans_matrix) 206 else:

    /opt/conda/lib/python3.7/site-packages/kornia/augmentation/augmentation.py in compute_transformation(self, input, params) 1063 1064 def compute_transformation(self, input: torch.Tensor, params: Dict[str, torch.Tensor]) -> torch.Tensor: -> 1065 transform: torch.Tensor = get_perspective_transform(params['src'].to(input), params['dst'].to(input)) 1066 return transform 1067

    /opt/conda/lib/python3.7/site-packages/kornia/geometry/transform/imgwarp.py in get_perspective_transform(src, dst) 281 282 # solve the system Ax = b --> 283 X, LU = _torch_solve_cast(b, A) 284 285 # create variable to return

    /opt/conda/lib/python3.7/site-packages/kornia/utils/helpers.py in _torch_solve_cast(input, A) 94 dtype = torch.float32 95 ---> 96 out1, out2 = torch.solve(input.to(dtype), A.to(dtype)) 97 98 return (out1.to(input.dtype), out2.to(input.dtype))

    RuntimeError: solve: MAGMA library not found in compilation. Please rebuild with MAGMA.```

    here is the kaggle code please fix it 😭 https://www.kaggle.com/shreeshaaithal/notebookf22d408364

    opened by shreesha345 0
  • PytorchStremReader failed reading zip archive

    PytorchStremReader failed reading zip archive

    Hi there

    I keep getting this issue when I try and run VQGAN + CLIP



    RuntimeError Traceback (most recent call last) in () 12 # Apply these settings and run 13 settings = clipit.apply_settings() ---> 14 clipit.do_init(settings) 15 clipit.do_run(settings)

    4 frames /usr/local/lib/python3.7/dist-packages/torch/serialization.py in init(self, name_or_buffer) 240 class _open_zipfile_reader(_opener): 241 def init(self, name_or_buffer) -> None: --> 242 super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer)) 243 244

    RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

    opened by MarnoJacobs13 3
Owner
Muhammad Fathy Rashad
20 years old ML engineer & full stack developer
Muhammad Fathy Rashad
An architecture that makes any doodle realistic, in any specified style, using VQGAN, CLIP and some basic embedding arithmetics.

Sketch Simulator An architecture that makes any doodle realistic, in any specified style, using VQGAN, CLIP and some basic embedding arithmetics. See

null 12 Dec 18, 2022
Making a music video with Wav2CLIP and VQGAN-CLIP

music2video Overview A repo for making a music video with Wav2CLIP and VQGAN-CLIP. The base code was derived from VQGAN-CLIP The CLIP embedding for au

Joel Jang | 장요엘 163 Dec 26, 2022
Traditional deepdream with VQGAN+CLIP and optical flow. Ready to use in Google Colab

VQGAN-CLIP-Video cat.mp4 policeman.mp4 schoolboy.mp4 forsenBOG.mp4

null 23 Oct 26, 2022
Just playing with getting VQGAN+CLIP running locally, rather than having to use colab.

Just playing with getting VQGAN+CLIP running locally, rather than having to use colab.

Nerdy Rodent 2.3k Jan 4, 2023
Zero-Shot Text-to-Image Generation VQGAN+CLIP Dockerized

VQGAN-CLIP-Docker About Zero-Shot Text-to-Image Generation VQGAN+CLIP Dockerized This is a stripped and minimal dependency repository for running loca

Kevin Costa 73 Sep 11, 2022
CLIP + VQGAN / PixelDraw

clipit Yet Another VQGAN-CLIP Codebase This started as a fork of @nerdyrodent's VQGAN-CLIP code which was based on the notebooks of @RiversWithWings a

dribnet 276 Dec 12, 2022
Streamlit Tutorial (ex: stock price dashboard, cartoon-stylegan, vqgan-clip, stylemixing, styleclip, sefa)

Streamlit Tutorials Install pip install streamlit Run cd [directory] streamlit run app.py --server.address 0.0.0.0 --server.port [your port] # http:/

Jihye Back 30 Jan 6, 2023
CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP

CLIP-GEN [简体中文][English] 本项目在萤火二号集群上用 PyTorch 实现了论文 《CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP》。 CLIP-GEN 是一个 Language-F

null 75 Dec 29, 2022
FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

XCL 191 Dec 31, 2022
Deep Text Search is an AI-powered multilingual text search and recommendation engine with state-of-the-art transformer-based multilingual text embedding (50+ languages).

Deep Text Search - AI Based Text Search & Recommendation System Deep Text Search is an AI-powered multilingual text search and recommendation engine w

null 19 Sep 29, 2022
Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, ...)

Image Classification Project Killer in PyTorch This repo is designed for those who want to start their experiments two days before the deadline and ki

null 349 Dec 8, 2022
CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

CLIP (Contrastive Language–Image Pre-training) Experiments (Evaluation) Model Dataset Acc (%) ViT-B/32 (Paper) CIFAR100 65.1 ViT-B/32 (Our) CIFAR100 6

Myeongjun Kim 52 Jan 7, 2023
Source code for models described in the paper "AudioCLIP: Extending CLIP to Image, Text and Audio" (https://arxiv.org/abs/2106.13043)

AudioCLIP Extending CLIP to Image, Text and Audio This repository contains implementation of the models described in the paper arXiv:2106.13043. This

null 458 Jan 2, 2023
BTC-Generator - BTC Generator With Python

Что такое BTC-Generator? Это генератор чеков всеми любимого @BTC_BANKER_BOT Для

DoomGod 3 Aug 24, 2022
Simple ONNX operation generator. Simple Operation Generator for ONNX.

sog4onnx Simple ONNX operation generator. Simple Operation Generator for ONNX. https://github.com/PINTO0309/simple-onnx-processing-tools Key concept V

Katsuya Hyodo 6 May 15, 2022
PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

PaddlePaddle Vision Transformers State-of-the-art Visual Transformer and MLP Models for PaddlePaddle ?? PaddlePaddle Visual Transformers (PaddleViT or

null 1k Dec 28, 2022
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

null 152 Jan 2, 2023
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models

LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models. Developers can reproduce these SOTA methods and build their own methods.

TuZheng 405 Jan 4, 2023
LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

LWCC: A LightWeight Crowd Counting library for Python LWCC is a lightweight crowd counting framework for Python. It wraps four state-of-the-art models

Matija Teršek 39 Dec 28, 2022