implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks

Overview

YOLOR

implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks

PWC

Unified Network

To reproduce the results in the paper, please use this branch.

Model Test Size APtest AP50test AP75test APStest APMtest APLtest batch1 throughput
YOLOR-P6 1280 52.6% 70.6% 57.6% 34.7% 56.6% 64.2% 49 fps
YOLOR-W6 1280 54.1% 72.0% 59.2% 36.3% 57.9% 66.1% 47 fps
YOLOR-E6 1280 54.8% 72.7% 60.0% 36.9% 58.7% 66.9% 37 fps
YOLOR-D6 1280 55.4% 73.3% 60.6% 38.0% 59.2% 67.1% 30 fps
YOLOv4-P5 896 51.8% 70.3% 56.6% 33.4% 55.7% 63.4% 41 fps
YOLOv4-P6 1280 54.5% 72.6% 59.8% 36.6% 58.2% 65.5% 30 fps
YOLOv4-P7 1536 55.5% 73.4% 60.8% 38.4% 59.4% 67.7% 16 fps

Installation

Docker environment (recommended)

Expand
# create the docker container, you can change the share memory size if you have more.
nvidia-docker run --name yolor -it -v your_coco_path/:/coco/ -v your_code_path/:/yolor --shm-size=64g nvcr.io/nvidia/pytorch:20.11-py3

# apt install required packages
apt update
apt install -y zip htop screen libgl1-mesa-glx

# pip install required packages
pip install seaborn thop

# install mish-cuda if you want to use mish activation
# https://github.com/thomasbrandon/mish-cuda
# https://github.com/JunnYu/mish-cuda
cd /
git clone https://github.com/JunnYu/mish-cuda
cd mish-cuda
python setup.py build install

# install pytorch_wavelets if you want to use dwt down-sampling module
# https://github.com/fbcotter/pytorch_wavelets
cd /
git clone https://github.com/fbcotter/pytorch_wavelets
cd pytorch_wavelets
pip install .

# go to code folder
cd /yolor

Colab environment

Expand
git clone https://github.com/WongKinYiu/yolor
cd yolor

# pip install required packages
pip install -qr requirements.txt

# install mish-cuda if you want to use mish activation
# https://github.com/thomasbrandon/mish-cuda
# https://github.com/JunnYu/mish-cuda
git clone https://github.com/JunnYu/mish-cuda
cd mish-cuda
python setup.py build install
cd ..

# install pytorch_wavelets if you want to use dwt down-sampling module
# https://github.com/fbcotter/pytorch_wavelets
git clone https://github.com/fbcotter/pytorch_wavelets
cd pytorch_wavelets
pip install .
cd ..

Prepare COCO dataset

Expand
cd /yolor
bash scripts/get_coco.sh

Prepare pretrained weight

Expand
cd /yolor
bash scripts/get_pretrain.sh

Testing

yolor_p6.pt

python test.py --data data/coco.yaml --img 1280 --batch 32 --conf 0.001 --iou 0.65 --device 0 --cfg cfg/yolor_p6.cfg --weights yolor_p6.pt --name yolor_p6_val

You will get the results:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.52510
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.70718
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.57520
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.37058
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.56878
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.66102
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.39181
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.65229
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.71441
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.57755
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.75337
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.84013

Training

Single GPU training:

python train.py --batch-size 8 --img 1280 1280 --data coco.yaml --cfg cfg/yolor_p6.cfg --weights '' --device 0 --name yolor_p6 --hyp hyp.scratch.1280.yaml --epochs 300

Multiple GPU training:

python -m torch.distributed.launch --nproc_per_node 2 --master_port 9527 train.py --batch-size 16 --img 1280 1280 --data coco.yaml --cfg cfg/yolor_p6.cfg --weights '' --device 0,1 --sync-bn --name yolor_p6 --hyp hyp.scratch.1280.yaml --epochs 300

Training schedule in the paper:

python -m torch.distributed.launch --nproc_per_node 8 --master_port 9527 train.py --batch-size 64 --img 1280 1280 --data data/coco.yaml --cfg cfg/yolor_p6.cfg --weights '' --device 0,1,2,3,4,5,6,7 --sync-bn --name yolor_p6 --hyp hyp.scratch.1280.yaml --epochs 300
python -m torch.distributed.launch --nproc_per_node 8 --master_port 9527 tune.py --batch-size 64 --img 1280 1280 --data data/coco.yaml --cfg cfg/yolor_p6.cfg --weights 'runs/train/yolor_p6/weights/last_298.pt' --device 0,1,2,3,4,5,6,7 --sync-bn --name yolor_p6-tune --hyp hyp.finetune.1280.yaml --epochs 450
python -m torch.distributed.launch --nproc_per_node 8 --master_port 9527 train.py --batch-size 64 --img 1280 1280 --data data/coco.yaml --cfg cfg/yolor_p6.cfg --weights 'runs/train/yolor_p6-tune/weights/epoch_424.pt' --device 0,1,2,3,4,5,6,7 --sync-bn --name yolor_p6-fine --hyp hyp.finetune.1280.yaml --epochs 450

Inference

yolor_p6.pt

python detect.py --source inference/images/horses.jpg --cfg cfg/yolor_p6.cfg --weights yolor_p6.pt --conf 0.25 --img-size 1280 --device 0

You will get the results:

horses

Citation

@article{wang2021you,
  title={You Only Learn One Representation: Unified Network for Multiple Tasks},
  author={Wang, Chien-Yao and Yeh, I-Hau and Liao, Hong-Yuan Mark},
  journal={arXiv preprint arXiv:2105.04206},
  year={2021}
}

Acknowledgements

Expand
Issues
  • How to reproduce results on YOLOR-S4-DWT

    How to reproduce results on YOLOR-S4-DWT

    I'm trying to reproduce results on YOLOR-S4-DWT. It's reported 37% AP on paper branch. However, after several training times, the results around 35.1 and 35.2 AP. I use below command, is there something I need to change?:

    python train.py --batch-size 32 --img 640 640 --data data/coco.yaml --cfg models/yolor-ssss-dwt.yaml --weights '' --device 0 --name yolor-ssss-dwt-baseline --hyp hyp.scratch.s.yaml --epochs 300

    opened by thanhnt-2658 7
  • Maximum number of classes that can be trained?

    Maximum number of classes that can be trained?

    What is the maximum number of classes that can be trained with YOLOR? if i have imagenet object localisation 1000 classes dataset. Would it be able to train those ?

    opened by hiteshhedwig 7
  • Error while resuming training

    Error while resuming training

    Hello, I run a training and stopped it before it ends. When I try to resume the training using python3 train.py --resume I got the following error: Traceback (most recent call last): File "train.py", line 537, in <module> train(hyp, opt, device, tb_writer, wandb) File "train.py", line 81, in train model = Darknet(opt.cfg).to(device) # create File "yolor/models/models.py", line 530, in __init__ self.module_defs = parse_model_cfg(cfg) File "yolor/utils/parse_config.py", line 13, in parse_model_cfg with open(path, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: '.cfg'

    I also try to run: python3 train.py --cfg my_cfg.cfg --resume but I got the same error.

    Then I noticed that in train.py, l.502 there is the following line: opt.cfg, opt.weights, opt.resume = '', ckpt, True So the cfg filename is set to '', I tried to modify the line this way: opt.weights, opt.resume = ckpt, True but still got the same error.

    Do you have any clue?

    opened by Va2sili 6
  • inference another models like yolor_W6,E6,D6 error

    inference another models like yolor_W6,E6,D6 error

    Detecting with yolor_p6 is fine.

    But, another models like 'yolor_W6,E6,D6'

    Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\KANG\anaconda3\envs\OD\lib\site-packages\torch\serialization.py", line 594, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "C:\Users\KANG\anaconda3\envs\OD\lib\site-packages\torch\serialization.py", line 853, in _load result = unpickler.load() ModuleNotFoundError: No module named 'models.yolo'

    I guess W6,E6,D6 is saved by torch 1.4. but now i using torch 1.7

    Is there any solution not downgrading torch?

    opened by ehdrndd 6
  • 大佬,bacth_size=1,Out of memory?

    大佬,bacth_size=1,Out of memory?

    大佬,您好,我在使用自定义数据集训练yolor_p6时出现 cuda out of memory,我把batch_size=1依然会出现。我很奇怪。 train command: python train.py --batch-size 1 --img 416 416 --data person.yaml --cfg cfg/yolor_p6.cfg --weights '' --device 2 --name yolor_p6 --hyp hyp.scratch.416.yaml --epochs 300 log image result image 是我哪步出错了吗?

    opened by crazybill-first 6
  • Detection differences between YOLO PyTorch frameworks?

    Detection differences between YOLO PyTorch frameworks?

    I recently used ultralytics YOLOv3 archived repository to convert darknet weights to pytorch weights. I then ran inference on a set of images. Then, I used this yolor repository with the converted YOLOv3 Pytorch weights (and cfg file) to run inference on the same dataset: it appears results are way better, detection is more accurate. I am wondering why results are better with this repository: what's the difference between these two detectors? How comes that I can run inference using YOLOv3 weights with a YOLOR repository? I assume YOLOR reads my cfg file and detect these are YOLOv3 weights and then run YOLOv3 inference on my images but why are the results better than with the YOLOv3 repo then?

    opened by Va2sili 5
  • What should I do when testing doesn't work with pycocotools?

    What should I do when testing doesn't work with pycocotools?

                   Class      Images     Targets           P           R      [email protected]
                   Class      Images     Targets           P           R      [email protected]
                   Class      Images     Targets           P           R      [email protected]
                   Class      Images     Targets           P           R      [email protected]
                   Class      Images     Targets           P           R      [email protected]
                   Class      Images     Targets           P           R      [email protected]
      [email protected]:.95: 100%|█| 18/18 [00:14<00:00,  1.89it/s]
                     all         548    3.88e+04       0.377       0.559       0.488       0.309
    Speed: 8.8/4.6/13.4 ms inference/NMS/total per 1280x1280 image at batch-size 32
    
    Evaluating pycocotools mAP... saving runs/test/yolor_p6_val8/best_ap_predictions.json...
    loading annotations into memory...
    ERROR: pycocotools unable to run: expected str, bytes or os.PathLike object, not list
    Results saved to runs/test/yolor_p6_val8
    

    This is what I get while testing. What should I do when this happens?

    opened by cnr0724 5
  • How to download the pretrained weights ?

    How to download the pretrained weights ?

    On running

    cd /yolor
    bash scripts/get_pretrain.sh
    

    The .pt file cannot be downloaded due to a google drive warning and instead of the weights it is an html file of the warning

    <!DOCTYPE html><html><head><title>Google Drive - Virus scan warning</title><meta http-equiv="content-type" content="text/html; charset=utf-8"/><style nonce="dRFVskmataaNG/kovcRLZg">/* Copyright 2022 Google Inc. All Rights Reserved. */
    .goog-inline-block{position:relative;display:-moz-inline-box;display:inline-block}* html .goog-inline-block{display:inline}*:first-child+html .goog-inline-block{display:inline}.goog-link-button{position:relative;color:#15c;text-decoration:underline;cursor:pointer}.goog-link-button-disabled{color:#ccc;text-decoration:none;cursor:default}body{color:#222;font:normal 13px/1.4 arial,sans-serif;margin:0}.grecaptcha-badge{visibility:hidden}.uc-main{padding-top:50px;text-align:center}#uc-dl-icon{display:inline-block;margin-top:16px;padding-right:1em;vertical-align:top}#uc-text{display:inline-block;max-width:68ex;text-align:left}.uc-error-caption,.uc-warning-caption{color:#222;font-size:16px}#uc-download-link{text-decoration:none}.uc-name-size a{color:#15c;text-decoration:none}.uc-name-size a:visited{color:#61c;text-decoration:none}.uc-name-size a:active{color:#d14836;text-decoration:none}.uc-footer{color:#777;font-size:11px;padding-bottom:5ex;padding-top:5ex;text-align:center}.uc-footer a{color:#15c}.uc-footer a:visited{color:#61c}.uc-footer a:active{color:#d14836}.uc-footer-divider{color:#ccc;width:100%}</style><link rel="icon" href="null"/></head><body><div class="uc-main"><div id="uc-dl-icon" class="image-container"><div class="drive-sprite-aux-download-file"></div></div><div id="uc-text"><p class="uc-warning-caption">Google Drive can't scan this file for viruses.</p><p class="uc-warning-subcaption"><span class="uc-name-size"><a href="/open?id=1WyzcN1-I0n8BoeRhi_xVt8C5msqdx_7k">yolor-p6.pt</a> (72M)</span> is too large for Google to scan for viruses. Would you still like to download this file?</p><form id="downloadForm" action="https://drive.google.com/uc?export=download&amp;confirm&amp;id=1WyzcN1-I0n8BoeRhi_xVt8C5msqdx_7k&amp;confirm=t" method="post"><input type="submit" id="uc-download-link" class="goog-inline-block jfk-button jfk-button-action" value="Download anyway"/></form></div></div><div class="uc-footer"><hr class="uc-footer-divider"></div></body></html>
    

    And this in turn leads to

    Traceback (most recent call last):
      File "test.py", line 302, in <module>
        test(opt.data,
      File "test.py", line 55, in test
        model = attempt_load(weights, map_location=device)  # load FP32 model
      File "/home/aayush/yolor/models/experimental.py", line 137, in attempt_load
        model.append(torch.load(w, map_location=map_location)['model'].float().fuse().eval())  # load FP32 model
      File "/home/aayush/.local/lib/python3.8/site-packages/torch/serialization.py", line 713, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/aayush/.local/lib/python3.8/site-packages/torch/serialization.py", line 920, in _legacy_load
        magic_number = pickle_module.load(f, **pickle_load_args)
    _pickle.UnpicklingError: invalid load key, '<'.
    

    When I manually copy paste the link for yolor_p6.pt in the browser

    https://drive.google.com/uc?export=download&id=1Tdn3yqpZ79X7R1Ql0zNlNScB1Dv9Fp7

    And a folder gets downloaded

    https://drive.google.com/drive/folders/18IoN5F94WjRzvappk_4RRf8j9GIoPKc7?usp=sharing

    I am not sure how to use this folder as the checkpoints ? Can anyone advise on how to download the checkpoints ?

    opened by Aayush-Jain01 4
  • Precision, Recall and mAP seems incoherent

    Precision, Recall and mAP seems incoherent

    Hello,

    I am training YOLOR-D6, I obtain precision, recall and mAP results from test.py with following command using paper branch: python3 test.py --weights ./runs/train/yolor-d6-1280size-multiGPU/weights/best.pt --img 1280 --verbose --data data/dtld_test.yaml --batch 32 --task test --conf 0.4 --iou 0.5

    Here is the result: image I think the mAP0.5 is very high with those recall values. Is there a mistake in metrics.py? Or do I have to run test.py with different options? Thank you.

    opened by yusiyoh 4
  • Training on custom dataset and labels

    Training on custom dataset and labels

    I have a dataset of my own which has 8 labels, completely different from the coco labels. I changed the data/coco.names and data/coco.yaml accordingly. But I get an index error:

    Traceback (most recent call last):
      File "train.py", line 537, in <module>
        train(hyp, opt, device, tb_writer, wandb)
      File "train.py", line 344, in train
        log_imgs=opt.log_imgs if wandb else 0)
      File "/home/ubuntu/yolor/test.py", line 226, in test
        plot_images(img, output_to_target(output, width, height), paths, f, names)  # predictions
      File "/home/ubuntu/yolor/utils/plots.py", line 164, in plot_images
        cls = names[cls] if names else cls
    IndexError: list index out of range
    

    I tried printing the detection classes and its in the range 0-79 i.e coco labels. But why is this happening when I completely changed the labels?

    Training command: python train.py --batch-size 1 --img 1280 1280 --data coco.yaml --cfg cfg/yolor_p6.cfg --weights yolor_p6.pt --device 0 --name yolor_p6_digit --hyp hyp.scratch.1280.yaml --epochs 5

    opened by devloper13 4
  • Issues on the pretrained model

    Issues on the pretrained model

    Hi,

    Great work! I use the main branch to train my model. I checked the pretrained yolor_p6.pt is okay. But could you provide more well-trained model for fine-tune? such as yolor_w6.pt, yolor_e6.pt....

    I noticed in your paper branch, you provide these model, yolor-w6.pt, yolor-e6.pt....But it may not be compatible with the main branch code with the following error:

    ModuleNotFoundError: No module named 'models.yolo'

    Hope to get your advice!

    opened by Yuuuuuuuuuuuuuuuuuummy 4
  • I have a question about test.py.

    I have a question about test.py.

    Test was driven by the script below. test Script python test.py --data data/data.yaml --cfg cfg/yolor_p6.cfg --weights runs/train/yolor_custom3/weights/best.pt --device 0 --img-size 512 --names data/data.names --conf 0.05

    The results show labels and pred in jpg format in runs/test/expXX.

    However, _label.jpg shows the correct bounding box, but _pred.jpg doesn't create a bounding box.

    Is it because prediction failed?

    The result of running detect.py is generating the correct bounding box.

    Please help.

    opened by cslee99 0
  • keep aspect ratio

    keep aspect ratio

    Hello!

    I have multiple similar objects, that have different aspect ratios. But for the rest they look pretty much the same.. I do get a lot of misinterpretations on those elements. Is there a way to tell yolor that during the training it should also keep track of the shape/proportion of the object? for isntance an ellypse is not a circle.. tagging them both in my dataset did not solve the issue. During training it keeps misdetecting them and the precision tops at about 0.78

    thank you

    opened by tanzerlana 0
  • W&B logging error of test.py occurs during training.

    W&B logging error of test.py occurs during training.

    An error occurs while learning. Please help. To train with custome data, a data.yaml file was created. Contents included are as follows.

    data.yaml train: E:/Dev/AI/yolor/data/roboflow/train/images val: E:/Dev/AI/yolor/data/roboflow/valid/images test: E:/Dev/AI/yolor/data/roboflow/test/images

    nc: 1 names: ['Stone']

    Train was driven by the script below. train Script python train.py --batch-size 8 --img 416 416 --data data/data.yaml --cfg cfg/yolor_p6.cfg --weights './yolor_p6.pt' --device 0 --name yolor_p6 --names data/data.names --hyp hyp.scratch.1280.yaml --epochs 50

    An error occurs at line 164 of test.py.

    Traceback (most recent call last): File "E:\Dev\AI\yolor\train.py", line 537, in train(hyp, opt, device, tb_writer, wandb) File "E:\Dev\AI\yolor\train.py", line 336, in train results, maps, times = test.test(opt.data, File "E:\Dev\AI\yolor\test.py", line 164, in test box_data = [{"position": {"minX": xyxy[0], "minY": xyxy[1], "maxX": xyxy[2], "maxY": xyxy[3]}, File "E:\Dev\AI\yolor\test.py", line 166, in "box_caption": "%s %.3f" % (names[cls], conf), TypeError: list indices must be integers or slices, not float

    The value of the cls variable is displayed as a float. (0.0, 23.0) Shouldn't it be changed in the code above? change code : name[cls] ==> name[int(cls)]

    Also, I have nc:1 and num of classes is 1, so why does it come out with values of 23.0 and 31.0?

    First, to see that it works perfectly, I changed the code to names[0] and proceeded. However, at line 170 of test.py, the following error occurs.

    wandb_images 'list' object has no attribute 'items'

    opened by cslee99 0
  • train new model ---yolor-p6.yaml

    train new model ---yolor-p6.yaml

    If I want to train a new model with yolor-p6, what do I need to do besides modifying "number of classes"? I didn't find a place to modify "filters=255" like yolor-p6.cfg. What should I do? Can you help me? QQ截图20220615212055 QQ截图20220615212706

    opened by qutyyds 2
  • strange custom train batch image

    strange custom train batch image

    Hello. I use yolor model for my object detection task. I trained my custom data with no argument anything. (no moasic, no flip, no translation, no rotation, ...)

    I think train batch images must be clear images like this. train_batch4

    But sometimes I got a strange batch images like this(about 10%). train_batch2

    Can I help my problem?

    I share my hyperparameter settings. lr0: 0.01 lrf: 0.2 momentum: 0.937 weight_decay: 0.001 warmup_epochs: 3.0 warmup_momentum: 0.8 warmup_bias_lr: 0.1 box: 0.05 cls: 0.5 cls_pw: 1.0 obj: 1.0 obj_pw: 1.0 iou_t: 0.2 anchor_t: 4.0 fl_gamma: 0.0 hsv_h: 0.0 hsv_s: 0.0 hsv_v: 0.0 degrees: 0.0 translate: 0.0 scale: 0.0 shear: 0.0 perspective: 0.0 flipud: 0.0 fliplr: 0.0 mosaic: 0.0 mixup: 0.0

    opened by jungyuko 0
Owner
Kin-Yiu, Wong
Kin-Yiu, Wong
Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Introduction Pytorch implementation of Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Expert. | paper Song Park1

Clova AI Research 83 Jun 15, 2022
Code for Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights

Piggyback: https://arxiv.org/abs/1801.06519 Pretrained masks and backbones are available here: https://uofi.box.com/s/c5kixsvtrghu9yj51yb1oe853ltdfz4q

Arun Mallya 158 May 26, 2022
Code for PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

PackNet: https://arxiv.org/abs/1711.05769 Pretrained models are available here: https://uofi.box.com/s/zap2p03tnst9dfisad4u0sfupc0y1fxt Datasets in Py

Arun Mallya 207 Jun 26, 2022
Compute execution plan: A DAG representation of work that you want to get done. Individual nodes of the DAG could be simple python or shell tasks or complex deeply nested parallel branches or embedded DAGs themselves.

Hello from magnus Magnus provides four capabilities for data teams: Compute execution plan: A DAG representation of work that you want to get done. In

null 12 Feb 8, 2022
ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representa

Bats Research 84 May 29, 2022
CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning This repository contains the code and relevant instructions

XiaoMing 5 May 4, 2022
Object detection on multiple datasets with an automatically learned unified label space.

Simple multi-dataset detection An object detector trained on multiple large-scale datasets with a unified label space; Winning solution of E

Xingyi Zhou 365 Jun 20, 2022
You Only Look One-level Feature (YOLOF), CVPR2021, Detectron2

You Only Look One-level Feature (YOLOF), CVPR2021 A simple, fast, and efficient object detector without FPN. This repo provides a neat implementation

qiang chen 260 Jun 25, 2022
You Only 👀 One Sequence

You Only ?? One Sequence TL;DR: We study the transferability of the vanilla ViT pre-trained on mid-sized ImageNet-1k to the more challenging COCO obje

Hust Visual Learning Team 589 Jun 21, 2022
A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

AMAZ3DSim AMAZ3DSim is a lightweight python-based 3D network multi-agent simulator. It uses a cell-based congestion model. It calculates risk, battery

Daniel Hirsch 11 Apr 12, 2022
This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.

Multimodal Deep Learning ?? ?? ?? Announcing the multimodal deep learning repository that contains implementation of various deep learning-based model

Deep Cognition and Language Research (DeCLaRe) Lab 279 Jun 22, 2022
Official implementation of the paper: "LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech"

LDNet Author: Wen-Chin Huang (Nagoya University) Email: [email protected] This is the official implementation of the paper "LDNet

Wen-Chin Huang (unilight) 32 Jun 15, 2022
Eff video representation - Efficient video representation through neural fields

Neural Residual Flow Fields for Efficient Video Representations 1. Download MPI

null 34 May 11, 2022
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

null 32 Jun 20, 2022
Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

USDAN The implementation of Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, which is accepte

null 7 Mar 12, 2022
This project uses reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can learn to read tape. The project is dedicated to hero in life great Jesse Livermore.

Reinforcement-trading This project uses Reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can

Deepender Singla 1.4k Jun 10, 2022
Game Agent Framework. Helping you create AIs / Bots that learn to play any game you own!

Serpent.AI - Game Agent Framework (Python) Update: Revival (May 2020) Development work has resumed on the framework with the aim of bringing it into 2

Serpent.AI 6.3k Jun 26, 2022
This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis Install the package in the requirements.txt, the

null 87 Jun 18, 2022
Code repo for EMNLP21 paper "Zero-Shot Information Extraction as a Unified Text-to-Triple Translation"

Zero-Shot Information Extraction as a Unified Text-to-Triple Translation Source code repo for paper Zero-Shot Information Extraction as a Unified Text

cgraywang 61 Jun 20, 2022