Code for "Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search"

Overview

Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search

This is an implementation for our paper Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search. The code is modified from Github repositoty "pytorch implementation for ECCV2018 paper Deep Cross-Modal Projection Learning for Image-Text Matching".

Requirement

  • Python 3.7
  • Pytorch 1.0.0 & torchvision 0.2.1
  • numpy
  • matplotlib (not necessary unless the need for the result figure)
  • scipy 1.2.1
  • pytorch_transformers

Usage

Data Preparation

  1. Please download CUHK-PEDES dataset .
  2. Put reid_raw.json under project_directory/data/
  3. run data.sh
  4. Copy files test_reid.json, train_reid.json and val_reid.json under CUHK-PEDES/data/ to project_directory/data/processed_data/
  5. Download pretrained Resnet50 model, bert-base-uncased model and vocabulary to project_directory/pretrained/

Training & Testing

You should firstly change the parameter BASE_ROOT to your current directory and IMAGE_DIR to the directory of CUHK-PEDES dataset. Run command sh scripts/train.sh to train the model. Run command sh scripts/test.sh to evaluate the model.

Model Framework

Framework

Model Performance

Performance0 Performance0

Comments
  • Confused about contrastive_loss

    Confused about contrastive_loss

    @girafffeee Thanks for your work! However, I am confused about what this loss function to do: https://github.com/TencentYoutuResearch/PersonReID-NAFS/blob/71bf192f464a410845a88590e4660eeeba1a1e50/utils/metric.py#L265-L286

    Why the criterionand t2i_similaritiesare not used in this function? Does this function do the same thing like CSAL in the paper?

    opened by BrandonHanx 6
  • bert-base-uncased-vocab.txt' not found in model shortcut name list

    bert-base-uncased-vocab.txt' not found in model shortcut name list

    I have downloaded pretrained Resnet50 model, bert-base-uncased model and vocabulary to project_directory/pretrained/.But when i run train.sh,it showed that '../pretrained/bert-base-uncased/bert-base-uncased-vocab.txt not found in model shortcut name list''.I attach the picture of the problem,please take a look at the problem. Is it a problem with the file directory structure?Great thanks if get your help! image

    opened by yyll1998 2
  • memory increased with steps

    memory increased with steps

    Hi all, @X-ing @girafffeee

    I found the memory increased with steps (about 20MB for several ten steps). This is not a good thing. Did you encounter the same phenomenon?'

    BTW, would you please take a little time to clean this repo? The code is a little bit messy (too many unused variables and imports, no lint and so on)...

    Thanks

    opened by BrandonHanx 2
  • Questions about training

    Questions about training

    Hello, thanks for sharing your code! :> I have two questions.

    1. I found out that in the uploaded training log, arguments given to "--part2" and "--part3" are 3 and 2 (not 2 and 3). Is it just a typo?

    2. As I have only TitanXP GPUs, when I run the code with 64 batches, I got 'runtime: CUDA out of memory' error. so I changed the batch size to 32 and it works. However, at 16 epochs suddenly loss increased and it did not decrease at all. Any idea about it? image I set the environment the same as you specified in the readme.

    opened by HaeyeonPark 1
  • Problems when run test.sh

    Problems when run test.sh

    First of all, thank you for sharing the code. But I have a bug running test.sh. I have a screenshot below. The bug is that best_model is being cast to an int. But I looked at model_path and there are only two files, model_best and best_model.pth.tar. There is nothing to convert to an int. Is there any problem with my file location? 1616390619(1)

    opened by ht374 1
  • Some confusion about the utilized jaccard distance in code

    Some confusion about the utilized jaccard distance in code

    Thanks for your code, but I have some confusion about the Jaccard distance API used in your testing code.

    https://github.com/TencentYoutuResearch/PersonReID-NAFS/blob/5301b8d95bb711f88357d71cff1a9aaaac76f473/utils/metric.py#L482

    As we can see in the document of SciPy, scipy.distance.jaccard is used to compute the Jaccard-Needham dissimilarity between two boolean 1-D arrays. It seems that it is not the expected implementation of the Jaccard Distance in your paper. Is it ok to directly use this API?

    opened by JonnieWayy 1
  • some questions about the backbone

    some questions about the backbone

    The last serveral layers of Resnet are not shared between the different scales, which seems like the final performance improvement mainly comes from random ensembling. May you report the results with the shared Resnet?

    opened by zifyloo 0
  • RuntimeError: shape '[-1, 2, 768]' is invalid for input of size 33024

    RuntimeError: shape '[-1, 2, 768]' is invalid for input of size 33024

    Please, what is the reason for this? File "/home/bobo/E/xxx/paper4_reid_2021/PersonReID-NAFS-main/models/model.py", line 76, in forward local_text_feat = local_text_feat.view(-1, n_sep, local_text_feat.size(1)) RuntimeError: shape '[-1, 2, 768]' is invalid for input of size 33024

    opened by liuliu408 1
  • Questions about the training log

    Questions about the training log

    Thanks for sharing your brilliant work! I see that in the train.log you shown, top1_t2i suddenly dropped to 0.244 when epoch was equal to 17, but top1_t2i returned to a normal value of 54.613 when epoch was equal to18. Could you please tell me what causes this problem in training?

    opened by ht374 0
  • question about performance

    question about performance

    I trained the model without any changes, with the default training strategy (set in run.sh), but obtained 54.2% R@1. Would you please tell me how to achieve the reported 61.5% R@1 performance as in paper? Should I change the training strategy or some hyper-parameters? Great thanks if get your help.

    opened by baixiao930 9
Owner
Tencent YouTu Research
Tencent YouTu Research
TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

null 73 Nov 6, 2022
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Oct 6, 2022
A code generator from ONNX to PyTorch code

onnx-pytorch Generating pytorch code from ONNX. Currently support onnx==1.9.0 and torch==1.8.1. Installation From PyPI pip install onnx-pytorch From

Wenhao Hu 94 Jan 6, 2023
This is the code for our KILT leaderboard submission to the T-REx and zsRE tasks. It includes code for training a DPR model then continuing training with RAG.

KGI (Knowledge Graph Induction) for slot filling This is the code for our KILT leaderboard submission to the T-REx and zsRE tasks. It includes code fo

International Business Machines 72 Jan 6, 2023
Convert Python 3 code to CUDA code.

Py2CUDA Convert python code to CUDA. Usage To convert a python file say named py_file.py to CUDA, run python generate_cuda.py --file py_file.py --arch

Yuval Rosen 3 Jul 14, 2021
Empirical Study of Transformers for Source Code & A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code

Transformers for variable misuse, function naming and code completion tasks The official PyTorch implementation of: Empirical Study of Transformers fo

Bayesian Methods Research Group 56 Nov 15, 2022
Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

This repository is a toolkit to do machine learning for programming languages. It implements tokenization, dataset preprocessing, model training and m

Facebook Research 408 Jan 1, 2023
Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Zhensu Sun 1 Oct 26, 2021
Low-code/No-code approach for deep learning inference on devices

EzEdgeAI A concept project that uses a low-code/no-code approach to implement deep learning inference on devices. It provides a componentized framewor

On-Device AI Co., Ltd. 7 Apr 5, 2022
Code for all the Advent of Code'21 challenges mostly written in python

Advent of Code 21 Code for all the Advent of Code'21 challenges mostly written in python. They are not necessarily the best or fastest solutions but j

null 4 May 26, 2022
Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

This codebase is being actively maintained, please create and issue if you have issues using it Basics All data files are included under losses and ea

J K Terry 32 Nov 9, 2021
Opinionated code formatter, just like Python's black code formatter but for Beancount

beancount-black Opinionated code formatter, just like Python's black code formatter but for Beancount Try it out online here Features MIT licensed - b

Launch Platform 16 Oct 11, 2022
a delightful machine learning tool that allows you to train, test and use models without writing code

igel A delightful machine learning tool that allows you to train/fit, test and use models without writing code Note I'm also working on a GUI desktop

Nidhal Baccouri 3k Jan 5, 2023
Pytorch Lightning code guideline for conferences

Deep learning project seed Use this seed to start new deep learning / ML projects. Built in setup.py Built in requirements Examples with MNIST Badges

Pytorch Lightning 1k Jan 2, 2023
Automatically Build Multiple ML Models with a Single Line of Code. Created by Ram Seshadri. Collaborators Welcome. Permission Granted upon Request.

Auto-ViML Automatically Build Variant Interpretable ML models fast! Auto_ViML is pronounced "auto vimal" (autovimal logo created by Sanket Ghanmare) N

AutoViz and Auto_ViML 397 Dec 30, 2022
Code samples for my book "Neural Networks and Deep Learning"

Code samples for "Neural Networks and Deep Learning" This repository contains code samples for my book on "Neural Networks and Deep Learning". The cod

Michael Nielsen 13.9k Dec 26, 2022
Code for: https://berkeleyautomation.github.io/bags/

DeformableRavens Code for the paper Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks. Here is the

Daniel Seita 121 Dec 30, 2022
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

Malik Boudiaf 138 Dec 12, 2022
Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

gtn_applications An applications library using GTN. Current examples include: Offline handwriting recognition Automatic speech recognition Installing

Facebook Research 68 Dec 29, 2022