Aligning Latent and Image Spaces to Connect the Unconnectable

Related tags

Deep Learning alis
Overview

About

This repo contains the official implementation of the Aligning Latent and Image Spaces to Connect the Unconnectable paper. It is a GAN model which can generate infinite images of diverse and complex scenes.

ALIS generation example

[Project page] [Paper]

Installation

To install, run the following command:

conda env create --file environment.yml --prefix ./env
conda activate ./env

Note: the tensorboard requirement is crucial, because otherwise upfirdn2d will not compile for some magical reason.

Training

To train the model, navigate to the project directory and run:

python infra/launch_local.py hydra.run.dir=. +experiment_name=my_experiment_name +dataset=dataset_name num_gpus=4

where dataset_name is the name of the dataset without .zip extension inside data/ directory (you can easily override the paths in configs/main.yml). So make sure that data/dataset_name.zip exists and should be a plain directory of images. See StyleGAN2-ADA repo for additional data format details. This training command will create an experiment inside experiments/ directory and will copy the project files into it. This is needed to isolate the code which produces the model.

Inference

The inference example can be found in notebooks/generate.ipynb

Data format

We use the same data format as the original StyleGAN2-ADA repo: it is a zip of images. It is assumed that all data is located in a single directory, specified in configs/main.yml. Put your datasets as zip archives into data/ directory.

Pretrained checkpoints

We provide checkpoints for the following datasets:

  • LHQ 1024x1024 with FID = 7.8. Note: this checkpoint has patch size of 1024x512, i.e. the image is generated in just 2 halves.

License

The project is based on the StyleGAN2-ADA repo developed by NVidia. I am not a lawyer, but I suppose that NVidia License applies to this project then.

Comments
  • Is Python 3.8.5 necessary?

    Is Python 3.8.5 necessary?

    Thanks for your interesting work!

    When running your generate.ipynb on Linux, I am trapped in a messy dependency issue. By reviewing your environment specification, I notice your Python version is quite high (for now): https://github.com/universome/alis/blob/9699eba112eda2ea27d6023221df2df9dc270b7f/environment.yml#L6

    For example, when I install opencv via conda, I get:

    Collecting package metadata (current_repodata.json): done
    Solving environment: failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
    Collecting package metadata (repodata.json): done
    Solving environment: failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: \ 
    Found conflicts! Looking for incompatible packages.
    This can take several minutes.  Press CTRL-C to abort.                                                                                                                                                                failed                                                                                                                                                                                                                    
    
    UnsatisfiableError: The following specifications were found
    to be incompatible with the existing python installation in your environment:
    
    Specifications:
    
      - opencv -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
    
    Your python: defaults/linux-64::python==3.8.5=h7579374_1
    
    If python is on the left-most side of the chain, that's the version you've asked for.
    When python appears to the right, that indicates that the thing on the left is somehow
    not available for the python version you are constrained to. Note that conda will not
    change your python version to a different minor version unless you explicitly specify
    that.
    

    I wanna know if this Python version is necessary since I would like to downgrade it.

    BTW, there's no opencv specified in your environment.yml.

    opened by BrandoZhang 6
  • How to resume training from provided checkpoint?

    How to resume training from provided checkpoint?

    I couldn't locate an option to specify the pretrained checkpoint so I specified it in training/training_loop.py in resume_pkl field. After running the training job, I encounter this error:

    -- Process 0 terminated with the following error:
    Traceback (most recent call last):
      File "/opt/conda/envs/alis/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
        fn(i, *args)
      File "/alis/experiments/shinkansen_slide2-b696bbb/scripts/train.py", line 443, in subprocess_fn
        training_loop.training_loop(rank=rank, **args)
      File "/alis/training/training_loop.py", line 177, in training_loop
        resume_data = legacy.load_network_pkl(f)
      File "/alis/experiments/shinkansen_slide2-b696bbb/scripts/legacy.py", line 22, in load_network_pkl
        data = _LegacyUnpickler(f).load()
      File "/alis/torch_utils/persistence.py", line 190, in _reconstruct_persistent_obj
        module = _src_to_module(meta.module_src)
      File "/alis/torch_utils/persistence.py", line 226, in _src_to_module
        exec(src, module.__dict__) # pylint: disable=exec-used
      File "<string>", line 25, in <module>
    ImportError: attempted relative import with no known parent package
    

    What is the correct way to specify a pretrained checkpoint or resume from an existing one?

    opened by maderix 6
  • Feedback from running inference on Win10

    Feedback from running inference on Win10

    I'm on conda 4.9.2, Windows 10 64bit

    In case anyone finds it useful, then: I had to run conda install -c anaconda cmake To get CMAKE into the environment, however, for some reason pip has failed to install ninja from source.

    So after activating the environment, I did the following steps which did not get executed due to the ninja build fail: conda install ninja==1.10.0 conda install tqdm==4.59.0 gitpython scikit-learn pip install gpustat pip install tensorboard==2.4.1 pip install -e . pip install omegaconf click

    I was then able to run the inference notebook and generate a lovely panoramic image

    To generate the video, I had to first run: conda install -c conda-forge opencv To get opencv into the environment

    The video was then generated and it looks beautiful. Cheers!

    opened by Norod 3
  • Issue while launching generate.py

    Issue while launching generate.py

    Hello.

    THanks for your code. When i'm trying to launch generate file i'm getting this error:

    Downloading https://vision-cair.s3.amazonaws.com/alis/lhq1024-snapshot.pkl ... done
    Setting up PyTorch plugin "bias_act_plugin"... Failed!
    /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/bias_act.py:50: UserWarning: Failed to build CUDA kernels for bias_act. Falling back to slow reference implementation. Details:
    
    Error building extension 'bias_act_plugin': [1/3] /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/bias_act.cu -o bias_act.cuda.o 
    FAILED: bias_act.cuda.o 
    /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/bias_act.cu -o bias_act.cuda.o 
    nvcc fatal   : Unknown option '-generate-dependencies-with-compile'
    [2/3] c++ -MMD -MF bias_act.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/bias_act.cpp -o bias_act.o 
    ninja: build stopped: subcommand failed.
    
      warnings.warn('Failed to build CUDA kernels for bias_act. Falling back to slow reference implementation. Details:\n\n' + str(sys.exc_info()[1]))
    Setting up PyTorch plugin "upfirdn2d_plugin"... Failed!
    /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.py:34: UserWarning: Failed to build CUDA kernels for upfirdn2d. Falling back to slow reference implementation. Details:
    
    Error building extension 'upfirdn2d_plugin': [1/3] /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output upfirdn2d.cuda.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cu -o upfirdn2d.cuda.o 
    FAILED: upfirdn2d.cuda.o 
    /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output upfirdn2d.cuda.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cu -o upfirdn2d.cuda.o 
    nvcc fatal   : Unknown option '-generate-dependencies-with-compile'
    [2/3] c++ -MMD -MF upfirdn2d.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cpp -o upfirdn2d.o 
    ninja: build stopped: subcommand failed.
    
      warnings.warn('Failed to build CUDA kernels for upfirdn2d. Falling back to slow reference implementation. Details:\n\n' + str(sys.exc_info()[1]))
    Traceback (most recent call last):
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1673, in _run_ninja_build
        env=env)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/subprocess.py", line 512, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/home/daddywesker/SingularityNet/MixingImages/alis/generate.py", line 67, in <module>
        img = G.synthesis(curr_ws, ws_context=curr_ws_context, left_borders_idx=curr_left_borders_idx, noise='const')
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/training/networks.py", line 1030, in forward
        x, img = block(x, img, cur_ws, ws_context=curr_ws_context, left_borders_idx=left_borders_idx, **block_kwargs)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/training/networks.py", line 903, in forward
        **layer_kwargs)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/training/networks.py", line 531, in forward
        w_lerp_multiplier=self.cfg.patchwise.w_lerp_multiplier,
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/misc.py", line 101, in decorator
        return fn(*args, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/training/networks.py", line 200, in patchwise_conv2d
        flip_weight=flip_weight)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/misc.py", line 101, in decorator
        return fn(*args, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/training/networks.py", line 222, in patchwise_op
        y = op(x, *args, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/misc.py", line 101, in decorator
        return fn(*args, **kwargs)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/conv2d_resample.py", line 139, in conv2d_resample
        x = upfirdn2d.upfirdn2d(x=x, f=f, padding=[px0+pxt,px1+pxt,py0+pyt,py1+pyt], gain=up**2, flip_filter=flip_filter)
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.py", line 163, in upfirdn2d
        if impl == 'cuda' and x.device.type == 'cuda' and _init():
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.py", line 32, in _init
        _plugin = custom_ops.get_plugin('upfirdn2d_plugin', sources=sources, extra_cuda_cflags=['--use_fast_math'])
      File "/home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/custom_ops.py", line 110, in get_plugin
        torch.utils.cpp_extension.load(name=module_name, verbose=verbose_build, sources=sources, **build_kwargs)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1091, in load
        keep_intermediates=keep_intermediates)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1302, in _jit_compile
        is_standalone=is_standalone)
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1407, in _write_ninja_file_and_build_library
        error_prefix=f"Error building extension '{name}'")
      File "/home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1683, in _run_ninja_build
        raise RuntimeError(message) from e
    RuntimeError: Error building extension 'upfirdn2d_plugin': [1/3] /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output upfirdn2d.cuda.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cu -o upfirdn2d.cuda.o 
    FAILED: upfirdn2d.cuda.o 
    /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output upfirdn2d.cuda.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cu -o upfirdn2d.cuda.o 
    nvcc fatal   : Unknown option '-generate-dependencies-with-compile'
    [2/3] c++ -MMD -MF upfirdn2d.o.d -DTORCH_EXTENSION_NAME=upfirdn2d_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/TH -isystem /home/daddywesker/anaconda3/envs/torch/lib/python3.7/site-packages/torch/include/THC -isystem /home/daddywesker/anaconda3/envs/torch/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /home/daddywesker/SingularityNet/MixingImages/alis/torch_utils/ops/upfirdn2d.cpp -o upfirdn2d.o 
    ninja: build stopped: subcommand failed.
    
    
    Process finished with exit code 1
    

    As a clarification, generate.py is just your pynb copied to regular python script file, it's just easier for me. Tried to search for this nvcc fatal : Unknown option '-generate-dependencies-with-compile' but currently got no clue.

    I'm using pytorch 1.8, cuda 10.1, Ubuntu 20. THanks in advance for the help.

    opened by DaddyWesker 2
  • Fix: upfirdn2d_plugin compilation issue.

    Fix: upfirdn2d_plugin compilation issue.

    Solve permission issue of upfirdn2d_plugin compilation. See https://github.com/NVlabs/stylegan2-ada-pytorch/issues/39 and https://github.com/pytorch/pytorch/commit/13013848d58a606da7377affeb416affb65f4a8a .

    opened by BrandoZhang 0
  • About reproducing

    About reproducing

    Hi,

    Thank you for your great work! Have you ever cleaned the lhq dataset? I used the lhq_1024_jpg dataset to reproduce the effect, and the FID can only reach 9. I tried to fine-tune on the open source model, but the initial FID was as high as 24.

    Best, JiKun

    opened by liujikun 4
  • Comments / Clarification of Generate.ipynb

    Comments / Clarification of Generate.ipynb

    Thank you for the great project! I'd like to ask for some clarification of how the generation process works in the generate.ipynb notebook. A bit of guidance / code comments in the third cell (starts with "num_frames") would be most appreciated. In particular, I'd like to:

    • Explicitly define the starting vector of an inference (rather than starting from a random location).
    • More carefully control the magnitude of each "shift" that produces the segments of the panorama. Where can I adjust how "far away" in latent space each step is?

    I'm asking for these clarifications because I'd like to make a different sort of animation. Rather than panning across a long panoramic strip, I'd like the strip as a whole to animate across a "latent walk" (something like shown here, but in which the animation could be of arbitrary width).

    Again, thank you for an excellent project, and for any guidence.

    opened by ksteinfe 1
Owner
Ivan Skorokhodov
Ivan Skorokhodov
Benchmark spaces - Benchmarks of how well different two dimensional spaces work for clustering algorithms

benchmark_spaces Benchmarks of how well different two dimensional spaces work fo

Bram Cohen 6 May 7, 2022
Diverse Image Captioning with Context-Object Split Latent Spaces (NeurIPS 2020)

Diverse Image Captioning with Context-Object Split Latent Spaces This repository is the PyTorch implementation of the paper: Diverse Image Captioning

Visual Inference Lab @TU Darmstadt 34 Nov 21, 2022
For auto aligning, cropping, and scaling HR and LR images for training image based neural networks

ImgAlign For auto aligning, cropping, and scaling HR and LR images for training image based neural networks Usage Make sure OpenCV is installed, 'pip

null 15 Dec 4, 2022
[NeurIPS 2021 Spotlight] Aligning Pretraining for Detection via Object-Level Contrastive Learning

SoCo [NeurIPS 2021 Spotlight] Aligning Pretraining for Detection via Object-Level Contrastive Learning By Fangyun Wei*, Yue Gao*, Zhirong Wu, Han Hu,

Yue Gao 139 Dec 14, 2022
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
Official pytorch implementation of DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces

DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces Minhyuk Sung*, Zhenyu Jiang*, Panos Achlioptas, Niloy J. Mitra, Leonidas

Zhenyu Jiang 21 Aug 30, 2022
The Habitat-Matterport 3D Research Dataset - the largest-ever dataset of 3D indoor spaces.

Habitat-Matterport 3D Dataset (HM3D) The Habitat-Matterport 3D Research Dataset is the largest-ever dataset of 3D indoor spaces. It consists of 1,000

Meta Research 62 Dec 27, 2022
High-Resolution Image Synthesis with Latent Diffusion Models

Latent Diffusion Models Requirements A suitable conda environment named ldm can be created and activated with: conda env create -f environment.yaml co

CompVis Heidelberg 5.6k Jan 4, 2023
Hcpy - Interface with Home Connect appliances in Python

Interface with Home Connect appliances in Python This is a very, very beta inter

Trammell Hudson 116 Dec 27, 2022
PyTorch implementation of: Michieli U. and Zanuttigh P., "Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations", CVPR 2021.

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations This is the official PyTorch implementation

Multimedia Technology and Telecommunication Lab 42 Nov 9, 2022
Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search

CLIP-GLaSS Repository for the paper Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search An in-browser demo is

Federico Galatolo 172 Dec 22, 2022
SLAMP: Stochastic Latent Appearance and Motion Prediction

SLAMP: Stochastic Latent Appearance and Motion Prediction Official implementation of the paper SLAMP: Stochastic Latent Appearance and Motion Predicti

Kaan Akan 34 Dec 8, 2022
Visualizer using audio and semantic analysis to explore BigGAN (Brock et al., 2018) latent space.

BigGAN Audio Visualizer Description This visualizer explores BigGAN (Brock et al., 2018) latent space by using pitch/tempo of an audio file to generat

Rush Kapoor 2 Nov 21, 2022
Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

null 37 Dec 3, 2022
Autolfads-tf2 - A TensorFlow 2.0 implementation of Latent Factor Analysis via Dynamical Systems (LFADS) and AutoLFADS

autolfads-tf2 A TensorFlow 2.0 implementation of LFADS and AutoLFADS. Installati

Systems Neural Engineering Lab 11 Oct 29, 2022
Face Identity Disentanglement via Latent Space Mapping [SIGGRAPH ASIA 2020]

Face Identity Disentanglement via Latent Space Mapping Description Official Implementation of the paper Face Identity Disentanglement via Latent Space

null 150 Dec 7, 2022
Navigating StyleGAN2 w latent space using CLIP

Navigating StyleGAN2 w latent space using CLIP an attempt to build sth with the official SG2-ADA Pytorch impl kinda inspired by Generating Images from

Mike K. 55 Dec 6, 2022
Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Face Identity Disentanglement via Latent Space Mapping - Implement in pytorch with StyleGAN 2 Description Pytorch implementation of the paper Face Ide

Daniel Roich 58 Dec 24, 2022