The AugNet Python module contains functions for the fast computation of image similarity.

Related tags

Deep Learning computer-vision deep-learning unsupervised-learning image-retrieval image-similarity
Overview
Logo

AugNet

AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation arxiv link

In our work, we propose AugNet, a new deep learning training paradigm to learn image features from a collection of unlabeled pictures. We develop a method to construct the similarities between pictures as distance metrics in the embedding space by leveraging the inter-correlation between augmented versions of samples. Our experiments demonstrate that the method is able to represent the image in low dimensional space and performs competitively in downstream tasks such as image classification and image similarity comparison. Moreover, unlike many deep-learning-based image retrieval algorithms, our approach does not require access to external annotated datasets to train the feature extractor, but still shows comparable or even better feature representation ability and easy-to-use characteristics.

Install

pip install imgsim

Usage

import imgsim
import cv2

vtr = imgsim.Vectorizer()

img0 = cv2.imread("img0.png")
img1 = cv2.imread("img1.png")

vec0 = vtr.vectorize(img0)
vec1 = vtr.vectorize(img1)

dist = imgsim.distance(vec0, vec1)
print("distance =", dist)

Image Comparision Examples:

Please download the STL10 dataset from: https://cs.stanford.edu/~acoates/stl10/ and put the files under "./data/stl10_binary".

Please download the pretrained model from: https://drive.google.com/file/d/1pV3EBZPDDc3z_YKdRJu6ZBF5yn_IHhsK/view?usp=sharing and put the pth file under "./models"

Run "res34_model_training_with_STL.py" if you would like to train your own model. Run "kmeans_demo.ipynb" to test with K-Means clustering.

The followings are some image comparison examples. The left most images are the queries. The rest images are the topK most similar images that the algorithm found from the dataset based on the distances between the embeddings to the queries'.

Paris6k

Reference: https://www.robots.ox.ac.uk/~vgg/data/parisbuildings/

Anime Illustrations:

Reference: https://www.kaggle.com/mylesoneill/tagged-anime-illustrations

Pokemons:

Reference: https://veekun.com/dex/downloads

Humans Sketchs:

Reference: http://cybertron.cg.tu-berlin.de/eitz/projects/classifysketch/

Welcome to cite our work:

@misc{chen2021augnet,
    title={AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation},
    author={Mingxiang Chen and Zhanguo Chang and Haonan Lu and Bitao Yang and Zhuang Li and Liufang Guo and Zhecheng Wang},
    year={2021},
    eprint={2106.06250},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

TODO:

  1. batch vectorization

  2. multiple gpu

You might also like...
Official codebase for Pretrained Transformers as Universal Computation Engines.

universal-computation Overview Official codebase for Pretrained Transformers as Universal Computation Engines. Contains demo notebook and scripts to r

Kevin Lu 210 Dec 28, 2022
Custom TensorFlow2 implementations of forward and backward computation of soft-DTW algorithm in batch mode.

Batch Soft-DTW(Dynamic Time Warping) in TensorFlow2 including forward and backward computation Custom TensorFlow2 implementations of forward and backw

null 19 Aug 30, 2022
Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

PyTorch code to reproduce LyDROO algorithm [1], which is an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. It applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems and solves each subproblem via DROO algorithm. It includes:

Liang HUANG 87 Dec 28, 2022
Sample and Computation Redistribution for Efficient Face Detection
Sample and Computation Redistribution for Efficient Face Detection

Introduction SCRFD is an efficient high accuracy face detection approach which initially described in Arxiv. Performance Precision, flops and infer ti

Sajjad Aemmi 13 Mar 5, 2022
SymPy-powered, Wolfram|Alpha-like answer engine totally in your browser, without backend computation

SymPy Beta SymPy Beta is a fork of SymPy Gamma. The purpose of this project is to run a SymPy-powered, Wolfram|Alpha-like answer engine totally in you

Liumeo 25 Dec 21, 2022
A pure PyTorch batched computation implementation of "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition"

A pure PyTorch batched computation implementation of "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition"

張致強 14 Dec 2, 2022
(Personalized) Page-Rank computation using PyTorch

torch-ppr This package allows calculating page-rank and personalized page-rank via power iteration with PyTorch, which also supports calculation on GP

Max Berrendorf 69 Dec 3, 2022
python-timbl, originally developed by Sander Canisius, is a Python extension module wrapping the full TiMBL C++ programming interface. With this module, all functionality exposed through the C++ interface is also available to Python scripts. Being able to access the API from Python greatly facilitates prototyping TiMBL-based applications.

README: python-timbl Authors: Sander Canisius, Maarten van Gompel Contact: [email protected] Web site: https://github.com/proycon/python-timbl/ pytho

Maarten van Gompel 16 Jan 16, 2022
The code of “Similarity Reasoning and Filtration for Image-Text Matching” [AAAI2021]
The code of “Similarity Reasoning and Filtration for Image-Text Matching” [AAAI2021]

SGRAF PyTorch implementation for AAAI2021 paper of “Similarity Reasoning and Filtration for Image-Text Matching”. It is built on top of the SCAN and C

Ronnie_IIAU 149 Dec 22, 2022
Comments
  • Vectorize a batch of images at the same time?

    Vectorize a batch of images at the same time?

    Hi! It took me a while to find this project. There are only a limited number of examples of using image vectorization to calculate similarities, and this project looks great, but there are also imperfections.

    I installed the package via pip and everything works fine, except that I can only vectorize one image at a time, which is inefficient. Can you provide a function so that we can vectorize a bunch of images with one call? Thank you!

    opened by oOHeavenOo 3
  • Questions about your paper

    Questions about your paper

    Hello,

    After reading your paper and trying your codes, I have several questions below.

    1. On page 5 of your paper, you wrote

    "In the above model, we use Resnet18 as the backbone and STL-10 unlabeled dataset (containing 100,000 images) as the training set."

    But why your repo has the res34_model_training_with_STL? Did a new experiment show that res34 outperformed res18?

    1. In table 6, you compared image retrieval quality among different image retrieval models and your model, did you also train all image retrieval models on Pokemon, Anime Face, and Human Sketch data?

    2. The model on ImageNet and the model on STL have two different output length (768 and 512), why did you specify different fully-connected layers on different dataset? Have you tried to make the feature vector smaller?

    3. why the loss is not compared each epoch? It looks like n_pics * n_samples is your "batch size". In each round, you picked n_pics original images and augmented n_samples times so your Xs.shape == [n_pics * n_samples, 3, 224, 224]. I see you compared the "cur loss" with best loss every five rounds, what's the purpose of this?

    opened by Kieran31 2
  • Question about training hyper parameters

    Question about training hyper parameters

    Hi,

    Actually, I have left a message somewhere in other closed issues. Would you please tell me how many epoches did you train your model? The paper said that training on 2 v100 gpu takes 20 hours. Since I do not have v100 gpu, I do not know when should I stop training. Would you please tell me this?

    opened by CoinCheung 1
  • Question about download link for pretrained model

    Question about download link for pretrained model

    Hi, thank you for your great work. It seems that the download link for pretrained model is expired. Would your mind update it? https://github.com/chenmingxiang110/AugNet/blob/caab160320ba3ec4ded45926245da1c23356e639/README.md?plain=1#L46

    And I want to know that the default model which loaded in imgsim, is trained on which dataset?

    opened by mrbulb 0
Owner
Ming
惊了
Ming
GitHub
TensorFlow Similarity is a python package focused on making similarity learning quick and easy.

TensorFlow Similarity is a python package focused on making similarity learning quick and easy.

null 912 Jan 8, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
Sharpened cosine similarity torch - A Sharpened Cosine Similarity layer for PyTorch

Sharpened Cosine Similarity A layer implementation for PyTorch Install At your c

Brandon Rohrer 203 Nov 30, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 4, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 5.7k Feb 12, 2021
An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing

SVM Données Une base d’images contient 490 images pour l’apprentissage (400 voitures et 90 bateaux), et encore 21 images pour fait des tests. Prétrait

Achraf Rahouti 3 Nov 30, 2021
This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

null 5 Jan 4, 2023
aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Bayesian Methods for Hackers Using Python and PyMC The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chap

Cameron Davidson-Pilon 25.1k Jan 2, 2023
Scientific Computation Methods in C and Python (Open for Hacktoberfest 2021)

Sci - cpy README is a stub. Do expand it. Objective This repository is meant to be a ready reference for scientific computation methods. Do ⭐ it if yo

Sandip Dutta 7 Oct 12, 2022
Deep learning with dynamic computation graphs in TensorFlow

TensorFlow Fold TensorFlow Fold is a library for creating TensorFlow models that consume structured data, where the structure of the computation graph

null 1.8k Dec 28, 2022