Official TensorFlow code for the forthcoming paper

Overview

arXiv PWC PWC License

~ Efficient-CapsNet ~

Are you tired of over inflated and overused convolutional neural networks? You're right! It's time for CAPSULES :)

This repository has been made for two primarly reasons:

  • open source the code (most of) developed during our "first-stage" research on capsules, summarized by the forthcoming article "Efficient-CapsNet: Capsule Network with Self-Attention Routing". The repository let you play with Efficient-CapsNet and let you set the base for your own experiments.
  • be an hub and a headlight in the cyberspace to spread to the machine learning comunity the intrinsic potential and value of capsule. However, albeit remarkable results achieved by capsule networks, we're fully aware that they're only limited to toy datasets. Nevertheless, there's a lot to make us think that with the right effort and collaboration of the scientific community, capsule based networks could really make a difference in the long run. For now, feel free to dive in our work :))

1.0 Getting Started

1.1 Installation

Python3 and Tensorflow 2.x are required and should be installed on the host machine following the official guide. Good luck with it!

  1. Clone this repository
    git clone https://github.com/EscVM/Efficient-CapsNet.git
  2. Install the required packages
    pip3 install -r requirements.txt

Peek inside the requirements file if you have everything already installed. Most of the dependencies are common libraries.

2.0 Efficient-CapsNet Notebooks

The repository provides two starting notebooks to make you confortable with our architecture. They all have the information and explanations to let you dive further in new research and experiments. The first one let you test Efficient-CapsNet over three different datasets. The repository is provided with some of the weights derived by our own experiments. On the other hand, the second one let you train the network from scratch. It's a very lightweight network so you don't need "Deep Mind" TPUs arsenal to train it. However, even if a GP-GPU is not compulsory, it's strongly suggested (No GPU, no deep learning, no party).

3.0 Original CapsNet Notebooks

It goes without saying that our work has been inspiered by Geoffrey Hinton and his article "Dynamic Routing Between Capsules". It's really an honor to build on his idea. Nevertheless, when we did our first steps in the capsule world, we were pretty disappointed in finding that all repositories/implementations were ultimately wrong in some aspects. So, we implemented everything from scratch, carefully following the original Sara's repository. However, our implementation, besides beeing written for the new TensorFlow 2 version, is much more easier and practical to use. Sara's one is really overcomplicated and too mazy that you can lost pretty easily.

As for the previous section we provide two notebooks, one for testing (weights have been derived from Sara's repository) and one for training.

Nevertheless, there's a really negative note (at least for us:)); as all other repositories that you can find on the web, also our one is not capable to achieve the scores reported in their paper. We really did our best, but there is no way to make the network achieve a score greater than 99.64% on MNIST. Exactly for this reason, weights provided in this repository are derived from their repository. Anyway, it's Geoffrey so we can excuse him.

4.0 Capsules Dimensions Perturbation Notebook

The network is trained with a reconstruction regularizer that is simply a fully connected network trained in conjuction with the main one. So, we can use it to visualize the inner capsules reppresentations. In particular, we should expect that a dimension of a digit capsule should learn to span the space of variations in the way digits of that class are instantiated. We can see what the individual dimensions represent by making use of the decoder network and injecting some noise to one of the dimensions of the main digit capsule layer that is predicting the class of the input.

So, we coded a practical notebook in which you can dynamically tweak whichever dimension you want of the capsule that is making the prediction (longest one).

Finally, if you don't have the necessary resources (GP-GPU holy grail) you can still try this interesting notebook out on Colab.

Citation

Use this bibtex if you enjoyed this repository and you want to cite it:

@article{mazzia2021efficient,
  title={Efficient-CapsNet: Capsule Network withSelf-Attention Routing},
  author={Mazzia, Vittorio and Salvetti, Francesco and Chiaberge, Marcello},
  year={2021},
  journal={arXiv preprint arXiv:2101.12491},
}
Comments
  • UnimplementedError:  Fused conv implementation does not support grouped convolutions for now.

    UnimplementedError: Fused conv implementation does not support grouped convolutions for now.

    I got the same error on OS X 12.1 as well as on Google CoLab Pro.

    OS X tf.version '2.5.0'

    CoLab tf.version 2.6.0

    ---------------------------------------------------------------------------
    UnimplementedError                        Traceback (most recent call last)
    <ipython-input-9-03c7f065910f> in <module>
    ----> 1 history = model_train.train(dataset, initial_epoch=0)
    
    ~/Efficient-CapsNet/models/model.py in train(self, dataset, initial_epoch)
        178         print('-'*30 + f'{self.model_name} train' + '-'*30)
        179 
    --> 180         history = self.model.fit(dataset_train,
        181           epochs=self.config[f'epochs'], steps_per_epoch=steps,
        182           validation_data=(dataset_val), batch_size=self.config['batch_size'], initial_epoch=initial_epoch,
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
       1181                 _r=1):
       1182               callbacks.on_train_batch_begin(step)
    -> 1183               tmp_logs = self.train_function(iterator)
       1184               if data_handler.should_sync:
       1185                 context.async_wait()
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
        887 
        888       with OptionalXlaContext(self._jit_compile):
    --> 889         result = self._call(*args, **kwds)
        890 
        891       new_tracing_count = self.experimental_get_tracing_count()
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
        948         # Lifting succeeded, so variables are initialized and we can run the
        949         # stateless function.
    --> 950         return self._stateless_fn(*args, **kwds)
        951     else:
        952       _, _, _, filtered_flat_args = \
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
       3021       (graph_function,
       3022        filtered_flat_args) = self._maybe_define_function(args, kwargs)
    -> 3023     return graph_function._call_flat(
       3024         filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access
       3025 
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
       1958         and executing_eagerly):
       1959       # No tape is watching; skip to running the function.
    -> 1960       return self._build_call_outputs(self._inference_function.call(
       1961           ctx, args, cancellation_manager=cancellation_manager))
       1962     forward_backward = self._select_forward_and_backward_functions(
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
        589       with _InterpolateFunctionError(self):
        590         if cancellation_manager is None:
    --> 591           outputs = execute.execute(
        592               str(self.signature.name),
        593               num_outputs=self._num_outputs,
    
    ~/anaconda3/envs/ai/lib/python3.8/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
         57   try:
         58     ctx.ensure_initialized()
    ---> 59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
         60                                         inputs, attrs, num_outputs)
         61   except core._NotOkStatusException as e:
    
    UnimplementedError:  Fused conv implementation does not support grouped convolutions for now.
    	 [[node Efficinet_CapsNet_Generator/Efficient_CapsNet/primary_caps/conv2d/BiasAdd (defined at /Users/davidlaxer/Efficient-CapsNet/utils/layers.py:129) ]] [Op:__inference_train_function_3601]
    
    Function call stack:
    train_function
    

    I also tried running on tensorflow-metal w/GPU ... machine unresponsive ... had to reboot.

    duplicate 
    opened by dbl001 2
  • Depthwise separable convolution incomplete?

    Depthwise separable convolution incomplete?

    Hi @EscVM,

    to me, it seems like the depth wise separable convolution implemented in https://github.com/EscVM/Efficient-CapsNet/blob/705449c/utils/layers.py#L123 lacks the point-wise convolution part?!

    Shouldn't there be a second Conv2D layer with F filters of kernel size (1,1)?

    Otherwise, you're doing a simple depth-wise convolution, right?

    I am referencing https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728 here btw. Not sure if you gave a reference for the depthwise separable convolution in your paper.

    question 
    opened by TobiasB22 2
  • Visualisation notebook throws UnimplementedError:  Fused conv implementation does not support grouped convolutions for now

    Visualisation notebook throws UnimplementedError: Fused conv implementation does not support grouped convolutions for now

    I am getting an error when running model_test.evaluate(mnist_dataset.X_test, mnist_dataset.y_test) in the dynamic visualisations notebook. The first time I installed the requirements there were a couple of warnings, about incompatible versions. I tried installing again and everything was then OK.

    The notebook runs fully on Colab, thanks, but I would like to get it working locally. Any ideas to try and fix this please?


    UnimplementedError Traceback (most recent call last) in ----> 1 model_test.evaluate(mnist_dataset.X_test, mnist_dataset.y_test)

    C:\DSAI\Dissertation\Efficient-CapsNet\models\model.py in evaluate(self, X_test, y_test) 94 acc = np.mean(acc) 95 else: ---> 96 y_pred, X_gen = self.model.predict(X_test) 97 acc = np.sum(np.argmax(y_pred, 1) == np.argmax(y_test, 1))/y_test.shape[0] 98 test_error = 1 - acc

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\keras\engine\training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing) 1627 for step in data_handler.steps(): 1628 callbacks.on_predict_batch_begin(step) -> 1629 tmp_batch_outputs = self.predict_function(iterator) 1630 if data_handler.should_sync: 1631 context.async_wait()

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\eager\def_function.py in call(self, *args, **kwds) 826 tracing_count = self.experimental_get_tracing_count() 827 with trace.Trace(self._name) as tm: --> 828 result = self._call(*args, **kwds) 829 compiler = "xla" if self._experimental_compile else "nonXla" 830 new_tracing_count = self.experimental_get_tracing_count()

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\eager\def_function.py in _call(self, *args, **kwds) 892 *args, **kwds) 893 # If we did not create any variables the trace we have is good enough. --> 894 return self._concrete_stateful_fn._call_flat( 895 filtered_flat_args, self._concrete_stateful_fn.captured_inputs) # pylint: disable=protected-access 896

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\eager\function.py in _call_flat(self, args, captured_inputs, cancellation_manager) 1916 and executing_eagerly): 1917 # No tape is watching; skip to running the function. -> 1918 return self._build_call_outputs(self._inference_function.call( 1919 ctx, args, cancellation_manager=cancellation_manager)) 1920 forward_backward = self._select_forward_and_backward_functions(

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\eager\function.py in call(self, ctx, args, cancellation_manager) 553 with _InterpolateFunctionError(self): 554 if cancellation_manager is None: --> 555 outputs = execute.execute( 556 str(self.signature.name), 557 num_outputs=self._num_outputs,

    ~\miniconda3\envs\dsai_py3.8\lib\site-packages\tensorflow\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 57 try: 58 ctx.ensure_initialized() ---> 59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, 60 inputs, attrs, num_outputs) 61 except core._NotOkStatusException as e:

    UnimplementedError: Fused conv implementation does not support grouped convolutions for now. [[node Efficinet_CapsNet_Generator/Efficient_CapsNet/primary_caps_2/conv2d/BiasAdd (defined at C:\DSAI\Dissertation\Efficient-CapsNet\utils\layers.py:129) ]] [Op:__inference_predict_function_3999]

    Function call stack: predict_function

    documentation 
    opened by msm1089 2
  • UnimplementedError: Fused conv implementation does not support grouped convolutions for now

    UnimplementedError: Fused conv implementation does not support grouped convolutions for now

    Thanks for sharing the code of your awesome work.

    I am trying to run the different .ipynb of your work. However, once I run the "1.2.1 Evaluate the model"

    model_test.evaluate(mnist_dataset.X_test, mnist_dataset.y_test)

    I get the following error:

    `UnimplementedError: Fused conv implementation does not support grouped convolutions for now. [[node sequential/conv2d/Relu (defined at :1) ]] [Op:__inference_train_function_1358]

    Function call stack: train_function`

    (I am running on CPU) I would be grateful for any kind of help that will solve this issue. Thank you.

    duplicate 
    opened by MFalam 1
  • Performance issues in utils/pre_process_mnist.py(P2)

    Performance issues in utils/pre_process_mnist.py(P2)

    Hello,I found a performance issue in the definition of generate_tf_data , utils/pre_process_mnist.py, dataset_train = dataset_train.map was called without num_parallel_calls. I think it will increase the efficiency of your program if you add this.

    Here is the documemtation of tensorflow to support this thing.

    Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

    enhancement 
    opened by DLPerf 1
  • Overfitting

    Overfitting

    It is always overfitting on my own dataset, even I try my best to simplify the model which only contains 2000+ parameters. I data is (3003001) pictures, 800+ trainsing samples. Anyone can guide me this? Thanks

    help wanted 
    opened by whongfeiHK 1
  • Error running code

    Error running code

    Code:

    from utils import AffineVisualizer, Dataset
    from models import EfficientCapsNet
    
    mnist_dataset = Dataset('MNIST', config_path='config.json') # only MNIST
    
    model_test = EfficientCapsNet('MNIST', mode='test', verbose=False)
    model_test.load_graph_weights()
    model_play = EfficientCapsNet('MNIST', mode='play', verbose=False)
    model_play.load_graph_weights()
    
    model_test.evaluate(mnist_dataset.X_test, mnist_dataset.y_test)`
    
    ERROR:
    ````------------------------------MNIST Evaluation------------------------------
    2021-05-02 16:05:36.390373: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
    2021-05-02 16:05:36.390888: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2095074999 Hz
    Traceback (most recent call last):
      File "test01.py", line 12, in <module>
        model_test.evaluate(mnist_dataset.X_test, mnist_dataset.y_test)
      File "/home/NETID/datle/css_586/Efficient-CapsNet-main/models/model.py", line 96, in evaluate
        y_pred, X_gen =  self.model.predict(X_test)
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1629, in predict
        tmp_batch_outputs = self.predict_function(iterator)
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 828, in __call__
        result = self._call(*args, **kwds)
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 895, in _call
        filtered_flat_args, self._concrete_stateful_fn.captured_inputs)  # pylint: disable=protected-access
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/eager/function.py", line 1919, in _call_flat
        ctx, args, cancellation_manager=cancellation_manager))
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/eager/function.py", line 560, in call
        ctx=ctx)
      File "/home/NETID/datle/css_586/venv/lib64/python3.6/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
        inputs, attrs, num_outputs)
    tensorflow.python.framework.errors_impl.UnimplementedError:  Fused conv implementation does not support grouped convolutions for now.
    	 [[node Efficinet_CapsNet_Generator/Efficient_CapsNet/primary_caps/conv2d/BiasAdd (defined at /home/NETID/datle/css_586/Efficient-CapsNet-main/utils/layers.py:129) ]] [Op:__inference_predict_function_2000]
    
    Function call stack:
    predict_function`
    
    duplicate 
    opened by le000043 1
  • performance problems

    performance problems

    Is there a problem with the code? The performance of the model may be abnormal. There may be some problems with the accuracy.

    Epoch 35/150 3750/3750 [==============================] - 142s 38ms/step - loss: 0.0191 - Original_CapsNet_loss: 0.0106 - Generator_loss: 0.0217 - Original_CapsNet_multiAccuracy: 0.5635 - val_loss: 0.0138 - val_Original_CapsNet_loss: 0.0056 - val_Generator_loss: 0.0209 - val_Original_CapsNet_multiAccuracy: 0.5458 Epoch 36/150 3750/3750 [==============================] - 141s 38ms/step - loss: 0.0184 - Original_CapsNet_loss: 0.0100 - Generator_loss: 0.0215 - Original_CapsNet_multiAccuracy: 0.5591 - val_loss: 0.0133 - val_Original_CapsNet_loss: 0.0051 - val_Generator_loss: 0.0209 - val_Original_CapsNet_multiAccuracy: 0.5900 Epoch 37/150 3750/3750 [==============================] - 142s 38ms/step - loss: 0.0181 - Original_CapsNet_loss: 0.0098 - Generator_loss: 0.0212 - Original_CapsNet_multiAccuracy: 0.5619 - val_loss: 0.0132 - val_Original_CapsNet_loss: 0.0052 - val_Generator_loss: 0.0204 - val_Original_CapsNet_multiAccuracy: 0.5344

    bug 
    opened by StephenTaylor1998 1
  • Output is always 1

    Output is always 1

    I'm running the following code to test my custom images, and here is the cude:

    import tensorflow as tf
    from utils import Dataset, plotImages, plotWrongImages
    from models import EfficientCapsNet
    import cv2, numpy as np
    
    img_path = "imgs/box_9.jpg"
    img_np = cv2.bitwise_not(cv2.imread(img_path, 0)) # invert conversion
    
    gpus = tf.config.experimental.list_physical_devices('GPU')
    tf.config.experimental.set_visible_devices(gpus[0], 'GPU')
    tf.config.experimental.set_memory_growth(gpus[0], True)
    
    # some parameters
    model_name = 'MULTIMNIST'
    custom_path = None # if you've trained a new model, insert here the full graph weights path
    
    dataset = Dataset(model_name, config_path='config.json')
    
    model_test = EfficientCapsNet(model_name, mode='test', verbose=True, custom_path=custom_path)
    
    model_test.load_graph_weights() # load graph weights (bin folder)
    
    model_test.evaluate(dataset.X_test, dataset.y_test) # if "smallnorb" use X_test_patch
    
    #not working with MultiMNIST
    
    shape = dataset.X_test[0].shape
    
    img_np = cv2.resize(img_np, shape[:2], interpolation=cv2.INTER_AREA)
    img_np = img_np[..., None]
    
    y_pred = model_test.predict(np.expand_dims(img_np, axis=0))[0] # if "smallnorb" use X_test_patch
    
    print(np.argmax(y_pred))
    print(y_pred)
    
    cv2.imshow('show', img_np)
    cv2.waitKey(0)
    

    And here is the result:

    [[0.3153154  0.956559   0.2567287  0.2726928  0.18593171 0.6188526
      0.14006346 0.44058013 0.36337793 0.15720315]]
    

    As it's shown, index 1 has highest value and it means the output is 1. But I tested 13 images and all has highest value in index 1 (output is always 1). What am I doing wrong?

    question 
    opened by freedom9393 1
  • Multiple Caps Layers

    Multiple Caps Layers

    Hi

    we are very impressed by your article and the improvement you implemented. In the article you wrote: "However, we adopt only two layers of capsules due to the relative simplicity of the dataset investigated" we are trying to use your caps implementation with our problem. our input is 40X40X40 and a more complicated image than MNIST, therefore we want to use more caps layers. should we simply duplicate the PrimaryCaps layers before the FCCaps layer?

    Thanks

    Yaniv.

    question 
    opened by yanivgilad 2
  • Performance issues in /utils (by P3)

    Performance issues in /utils (by P3)

    Hello! I've found a performance issue in /utils: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.

    Detailed description is listed below:

    • /pre_process_mnist.py: dataset_train.batch(batch_size)(here) should be called before dataset_train.map(image_rotate_random,num_parallel_calls=PARALLEL_INPUT_CALLS)(here), dataset_train.map(image_shift_rand,num_parallel_calls=PARALLEL_INPUT_CALLS)(here), dataset_train.map(image_squish_random,num_parallel_calls=PARALLEL_INPUT_CALLS)(here), dataset_train.map(image_erase_random,num_parallel_calls=PARALLEL_INPUT_CALLS)(here) and dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)(here).
    • /pre_process_mnist.py: dataset_test.batch(batch_size)(here) should be called before dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)(here).
    • /pre_process_smallnorb.py: dataset_train.batch(batch_size)(here) should be called before dataset_train.map(random_patches,num_parallel_calls=PARALLEL_INPUT_CALLS)(here), dataset_train.map(random_brightness,num_parallel_calls=PARALLEL_INPUT_CALLS)(here), dataset_train.map(random_contrast,num_parallel_calls=PARALLEL_INPUT_CALLS)(here) and dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)(here).
    • /pre_process_smallnorb.py: dataset_test.batch(1)(here) should be called before dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)(here).

    Besides, you need to check the function called in map()(e.g., generator called in dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)) whether to be affected or not to make the changed code work properly. For example, if generator needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

    Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

    enhancement 
    opened by DLPerf 1
Owner
Vittorio Mazzia
Ph.D. Student in Machine Learning and Artificial Intelligence
Vittorio Mazzia
TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

null 73 Nov 6, 2022
The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Action Transformer A Self-Attention Model for Short-Time Human Action Recognition This repository contains the official TensorFlow implementation of t

PIC4SeRCentre 20 Jan 3, 2023
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy.

Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy. Now with tensorflow 1.0 support. Evaluation usa

Marcel R. 349 Aug 6, 2022
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

null 2.6k Jan 4, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Peter Lin 6.5k Jan 4, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting (RVM) English | 中文 Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specific

flow-dev 2 Aug 21, 2022
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

João Fonseca 3 Jan 3, 2023
StyleGAN2 - Official TensorFlow Implementation

StyleGAN2 - Official TensorFlow Implementation

NVIDIA Research Projects 10.1k Dec 28, 2022
Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

M-LSD: Towards Light-weight and Real-time Line Segment Detection Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line

NAVER/LINE Vision 357 Jan 4, 2023
An official TensorFlow implementation of “CLCC: Contrastive Learning for Color Constancy” accepted at CVPR 2021.

CLCC: Contrastive Learning for Color Constancy (CVPR 2021) Yi-Chen Lo*, Chia-Che Chang*, Hsuan-Chao Chiu, Yu-Hao Huang, Chia-Ping Chen, Yu-Lin Chang,

Yi-Chen (Howard) Lo 58 Dec 17, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
a reimplementation of UnFlow in PyTorch that matches the official TensorFlow version

pytorch-unflow This is a personal reimplementation of UnFlow [1] using PyTorch. Should you be making use of this work, please cite the paper according

Simon Niklaus 134 Nov 20, 2022
StyleGAN - Official TensorFlow Implementation

StyleGAN — Official TensorFlow Implementation Picture: These people are not real – they were produced by our generator that allows control over differ

NVIDIA Research Projects 13.1k Jan 9, 2023
StyleGAN2 with adaptive discriminator augmentation (ADA) - Official TensorFlow implementation

StyleGAN2 with adaptive discriminator augmentation (ADA) — Official TensorFlow implementation Training Generative Adversarial Networks with Limited Da

NVIDIA Research Projects 1.7k Dec 29, 2022
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Oct 6, 2022
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021
Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences", CVPR 2021.

HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature fo

Google Interns 50 Dec 21, 2022
Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Fastformer-Keras Unofficial Tensorflow-Keras implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Tensorflo

Yam Peleg 10 Jan 30, 2022