PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks

Overview

PICK-PyTorch

***** Updated on Feb 6th, 2021: Train Ticket dataset is now available for academic research. You can download from Google Drive or OneDrive. It contains 1,530 synthetic images and 320 real images for training, and 80 real images for testing. Please refer to our paper for more details about how to sample training/testing set from EATEN and generate the corresponding annotations.*****

***** Updated on Sep 17th, 2020: A training example on the large-scale document understanding dataset, DocBank, is now available. Please refer to examples/DocBank/README.md for more details. Thanks TengQi Ye for this contribution.*****

PyTorch reimplementation of "PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks" (ICPR 2020). This project is different from our original implementation.

Introduction

PICK is a framework that is effective and robust in handling complex documents layout for Key Information Extraction (KIE) by combining graph learning with graph convolution operation, yielding a richer semantic representation containing the textual and visual features and global layout without ambiguity. Overall architecture shown follows.

Overall

Requirements

  • python = 3.6
  • torchvision = 0.6.1
  • tabulate = 0.8.7
  • overrides = 3.0.0
  • opencv_python = 4.3.0.36
  • numpy = 1.16.4
  • pandas = 1.0.5
  • allennlp = 1.0.0
  • torchtext = 0.6.0
  • tqdm = 4.47.0
  • torch = 1.5.1
pip install -r requirements.txt

Usage

Distributed training with config files

Modify the configurations in config.json and dist_train.sh files, then run:

bash dist_train.sh

The application will be launched via launch.py on a 4 GPU node with one process per GPU (recommend).

This is equivalent to

python -m torch.distributed.launch --nnodes=1 --node_rank=0 --nproc_per_node=4 \
--master_addr=127.0.0.1 --master_port=5555 \
train.py -c config.json -d 1,2,3,4 --local_world_size 4

and is equivalent to specify indices of available GPUs by CUDA_VISIBLE_DEVICES instead of -d args

CUDA_VISIBLE_DEVICES=1,2,3,4 python -m torch.distributed.launch --nnodes=1 --node_rank=0 --nproc_per_node=4 \
--master_addr=127.0.0.1 --master_port=5555 \
train.py -c config.json --local_world_size 4

Similarly, it can be launched with a single process that spans all 4 GPUs (if node has 4 available GPUs) using (don't recommend):

CUDA_VISIBLE_DEVICES=1,2,3,4 python -m torch.distributed.launch --nnodes=1 --node_rank=0 --nproc_per_node=1 \
--master_addr=127.0.0.1 --master_port=5555 \
train.py -c config.json --local_world_size 1

Using Multiple Node

You can enable multi-node multi-GPU training by setting nnodes and node_rank args of the commandline line on every node. e.g., 2 nodes 4 gpus run as follows

Node 1, ip: 192.168.0.10, then run on node 1 as follows

CUDA_VISIBLE_DEVICES=1,2,3,4 python -m torch.distributed.launch --nnodes=2 --node_rank=0 --nproc_per_node=4 \
--master_addr=192.168.0.10 --master_port=5555 \
train.py -c config.json --local_world_size 4  

Node 2, ip: 192.168.0.15, then run on node 2 as follows

CUDA_VISIBLE_DEVICES=2,4,6,7 python -m torch.distributed.launch --nnodes=2 --node_rank=1 --nproc_per_node=4 \
--master_addr=192.168.0.10 --master_port=5555 \
train.py -c config.json --local_world_size 4  

Resuming from checkpoints

You can resume from a previously saved checkpoint by:

python -m torch.distributed.launch --nnodes=1 --node_rank=0 --nproc_per_node=4 \
--master_addr=127.0.0.1 --master_port=5555 \
train.py -d 1,2,3,4 --local_world_size 4 --resume path/to/checkpoint

Debug mode on one GPU/CPU training with config files

This option of training mode can debug code without distributed way. -dist must set to false to turn off distributed mode. -d specify which one gpu will be used.

python train.py -c config.json -d 1 -dist false

Testing from checkpoints

You can test from a previously saved checkpoint by:

python test.py --checkpoint path/to/checkpoint --boxes_transcripts path/to/boxes_transcripts \
               --images_path path/to/images_path --output_folder path/to/output_folder \
               --gpu 0 --batch_size 2

Customization

Training custom datasets

You can train your own datasets following the steps outlined below.

  1. Prepare the correct format of files as provided in data folder.
    • Please see data/README.md an instruction how to prepare the data in required format for PICK.
  2. Modify train_dataset and validation_dataset args in config.json file, including files_name, images_folder, boxes_and_transcripts_folder, entities_folder, iob_tagging_type and resized_image_size.
  3. Modify Entities_list in utils/entities_list.py file according to the entity type of your dataset.
  4. Modify keys.txt in utils/keys.txt file if needed according to the vocabulary of your dataset.
  5. Modify MAX_BOXES_NUM and MAX_TRANSCRIPT_LEN in data_tuils/documents.py file if needed.

Note: The self-build datasets our paper used cannot be shared for patient privacy and proprietary issues.

Checkpoints

You can specify the name of the training session in config.json files:

"name": "PICK_Default",
"run_id": "test"

The checkpoints will be saved in save_dir/name/run_id_timestamp/checkpoint_epoch_n, with timestamp in mmdd_HHMMSS format.

A copy of config.json file will be saved in the same folder.

Note: checkpoints contain:

{
  'arch': arch,
  'epoch': epoch,
  'state_dict': self.model.state_dict(),
  'optimizer': self.optimizer.state_dict(),
  'monitor_best': self.monitor_best,
  'config': self.config
}

Tensorboard Visualization

This project supports Tensorboard visualization by using either torch.utils.tensorboard or TensorboardX.

  1. Install

    If you are using pytorch 1.1 or higher, install tensorboard by 'pip install tensorboard>=1.14.0'.

    Otherwise, you should install tensorboardx. Follow installation guide in TensorboardX.

  2. Run training

    Make sure that tensorboard option in the config file is turned on.

     "tensorboard" : true
    
  3. Open Tensorboard server

    Type tensorboard --logdir saved/log/ at the project root, then server will open at http://localhost:6006

By default, values of loss will be logged. If you need more visualizations, use add_scalar('tag', data), add_image('tag', image), etc in the trainer._train_epoch method. add_something() methods in this project are basically wrappers for those of tensorboardX.SummaryWriter and torch.utils.tensorboard.SummaryWriter modules.

Note: You don't have to specify current steps, since WriterTensorboard class defined at logger/visualization.py will track current steps.

Results on Train Ticket

example

TODOs

  • Dataset cache mechanism to speed up training loop
  • Multi-node multi-gpu setup (DistributedDataParallel)

Citations

If you find this code useful please cite our paper:

@inproceedings{Yu2020PICKPK,
  title={{PICK}: Processing Key Information Extraction from Documents using 
  Improved Graph Learning-Convolutional Networks},
  author={Wenwen Yu and Ning Lu and Xianbiao Qi and Ping Gong and Rong Xiao},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  year={2020}
}

License

This project is licensed under the MIT License. See LICENSE for more details.

Acknowledgements

This project structure takes example by PyTorch Template Project.

Comments
  • 重新全文序列标注损失了OCR已经区分开的词组信息?

    重新全文序列标注损失了OCR已经区分开的词组信息?

    看了下最后输出关键词及类别的过程,好像是将全部文字组成了一个长序列,然后在里头提取出了各字的标签,再将连续同标签的字组成知名并打个此标签。这个过程中OCR部分识别为一个box里的词组的前后部分都可能被分配到其他组。 比如OCR出了三块"XXX","YYY","ZZZ“,序列标注的拆分结果可能变成了"XX","XYY","YZZ","Z"。 假设OCR的BOX组合的过程准确率很高,那如何更好利用这部分信息?

    opened by jingmouren 9
  • Move entities_list to config file; Add an option to specify image extension

    Move entities_list to config file; Add an option to specify image extension

    This PR consists from 3 commits:

    1. Change absolute path to the example dataset in config file to the relative one, so that train.py can work from scratch without any need to modify config file.

    2. Move entities_list to config file, so that all configuration steps for custom datasets can be done through config file, without any need to change code.

    Detailed commit summary

    Refactored parts of code that uses old version of Entities_List in order to make it able to work with entities_list that is modified after config file is parsed, during runtime.

    Global variables keys_vocab_cls, iob_labels and entities_vocab_cls defined utils/class_utils.py were moved to the global dict vocal_cls in order to make it possible to reassign keys_vocab_cls, iob_labels and entities_vocab_cls global variables without losing reference to them.

    File utils/entities.py was removed from the project, since entities_list is already declared in config.json.

    Part with entities_list for custom datasets in README.md was updated.

    Entities list was added to config.json in order to make training on example dataset work from scratch.

    3. Add an option to specify image extension to config.json file.

    Detailed commit summary

    Added image extension to config file, so that custom datasets with image extension that differs from .jpg can be configurated from config file. Supported extensions are:.jpg and .png.

    Updated README.md specifies the option to change the image extension.

    All tests successfully passed. Training on example dataset converges as before.

    opened by dbobrenko 8
  • result when run test.py

    result when run test.py

    Tks for your repo, the code is very clean. I trained your model with my dataset, the loss is converged good. But when predicted, a few the value of item['text'] was different format with the value of file .tsv. Example: "ÁO MOA TT CÀI CÚC" vs "ÁO MƯA TT CÀI CÚC",..vv Can you help me explain?

    opened by cuongngm 7
  • Testing with out-of-sample

    Testing with out-of-sample

    While testing with out-of-sample images, i tried creating boxes_and_transcripts for new images using tesseract & prepare tsv files. but when i'm predicting output is not good as in-sample testing images.

    kindly please let us know way to create bounding box files (tsv) for prediction for this model.

    Also can we predict without tsv files?

    Note: In tesseract i tried both WORD & TEXTLINE bounding boxes

    opened by karthikesh2020 7
  • Output files are empty when testing

    Output files are empty when testing

    Hello, thank you so much for providing code for great research paper. I am just wondering after I run python train.py --config config.json. All the values of mEP, mER, mEF, mEA are all 0 for every train Epoch. Is this normal? 2020-07-26 20:45:35.357166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 [2020-07-26 20:45:40,891 - train - INFO] - Model trainable parameters: 68575842 [2020-07-26 20:45:40,892 - train - INFO] - Train datasets: 8 samples Validation datasets: 8 samples Max_epochs: 100 Log_per_step: 10 Validation_per_step: 50 [2020-07-26 20:45:40,892 - train - INFO] - Training start... [2020-07-26 20:46:07,680 - trainer - INFO] - Train Epoch:[1/100] Step:[2/2] Total Loss: 1121.992798 GL_Loss: 8.390446 CRF_Loss: 1113.602295 [2020-07-26 20:46:09,988 - trainer - INFO] - [Epoch Validation] Epoch:[1/100] Total Loss: 1430.095032 GL_Loss: 0.091890 CRF_Loss: 1420.906006 +---------+-------+-------+-------+-------+ | name | mEP | mER | mEF | mEA | +=========+=======+=======+=======+=======+ | date | 0 | 0 | 0 | 0 | +---------+-------+-------+-------+-------+ | name | 0 | 0 | 0 | 0 | +---------+-------+-------+-------+-------+ | overall | 0 | 0 | 0 | 0 | +---------+-------+-------+-------+-------+ Also, I ran

    python test.py --checkpoint /content/PICK-pytorch/saved/models/PICK_Default/test2_0726_194721/model_best.pth --boxes_transcripts /content/PICK-pytorch/data/test_data_example/boxes_and_transcripts \
                   --images_path /content/PICK-pytorch/data/test_data_example/images/ --output_folder /content/PICK-pytorch/output/test2 \
                   --gpu 0 --batch_size 2
    

    Output files are generated, but the files are actually empty. Can you please give me the right direction to test the model? Thank you!

    opened by sophiajwchoi 6
  • Prediction

    Prediction

    I really like your PICK project and want to thank you for sharing it, also I have a question about the code.

    I am currently trying to export the prediction results from a trained model of PICK using the test.py script, but it just shows the predictions defined on the transcript of the .tsv file of the image provided, it doesn't export the other predictions, so my question is:  Where are the predictions stored? I tried looking through the code but didn't manage to find the variable that contains them. I would be very grateful if you could give some insight about this.

    A funny thing is that the script test.py just prints whatever I put in the .tsv, I replaced the transcripts of a .tsv with "asda sdas" and on the output folder got the same "asda sdas" on the .txt

    Regards.

    opened by compadrejavo 5
  • pretrained weights of docbank

    pretrained weights of docbank

    @wenwenyu @tengerye Hey can you please upload pretrained weights of docbank dataset. Because this dataset is huge and i am using colab so i am not able to retrain this one. If you able to do it please it would be very helpful.

    opened by kbrajwani 5
  • Error in class_utils.py:  TypeError: __init__() got an unexpected keyword argument 'specials'

    Error in class_utils.py: TypeError: __init__() got an unexpected keyword argument 'specials'

    When trying to run train.py for custom dataset, I got a type error. The custom dataset has been prepared properly as mentioned in the description. I am using Windows OS and my python version is 3.9 . Please help with this error. here is the traceback:

    Traceback (most recent call last): File "d:\PICK-pytorch\train.py", line 14, in
    import model.pick as pick_arch_module File "d:\PICK-pytorch\model\pick.py", line 12, in from .graph import GLCN File "d:\PICK-pytorch\model\graph.py", line 13, in from data_utils import documents File "d:\PICK-pytorch\data_utils\documents.py", line 16, in from utils.entities_list import Entities_list File "d:\PICK-pytorch\utils_init_.py", line 3, in from .util import * File "d:\PICK-pytorch\utils\util.py", line 11, in from .class_utils import keys_vocab_cls, iob_labels_vocab_cls File "d:\PICK-pytorch\utils\class_utils.py", line 62, in keys_vocab_cls = ClassVocab(classes=Path(file).parent.joinpath('keys.txt'))#, specials_first=False) File "d:\PICK-pytorch\utils\class_utils.py", line 44, in init super().init(c,specials=specials,**kwargs) TypeError: init() got an unexpected keyword argument 'specials'

    Thanks in advance for your responses.

    opened by nitinajithkumar 4
  • DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules

    DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules

    Hi, I encountered a problem with your code:

    AssertionError: DistributedDataParallel device_ids and output_device arguments only work
    with single-device CUDA modules, but got device_ids [0], output_device 0, and
    module parameters {device(type='cuda', index=0), device(type='cpu')}.
    

    But if I commented

    if self.config['trainer']['sync_batch_norm']:
        self.model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(self.model)
    

    of your trainer.py, the code runs without errors. I used the recommended settings. May I ask why?

    opened by tengerye 4
  • Test.py output UTF-8

    Test.py output UTF-8

    I am using text in Spanish, and when I test the dataset it does not return the Ñ correctly, I have searched through all the code and I cannot find where the line is to avoid this. Any help?

    The output:

    Address EGAOA Address ESPAOA

    It should be:

    Address EGAÑA Address ESPAÑA

    Thanks

    opened by jorgerodriguezsj 3
  • Information extraction when bounding boxes are not present (prediction on unseen data)

    Information extraction when bounding boxes are not present (prediction on unseen data)

    Hi,

    I was just exploring this model and found it useful for information extraction from scanned images. I need to know how can we make predictions on unlabelled data after we have trained the model?

    By unlabelled data, I mean images where we do not have bounding box details.

    Thanks in advance.

    opened by AbhayPadda 3
  • Information regarding Testing/ Inference

    Information regarding Testing/ Inference

    Hi @wenwenyu , please provide some information regarding inference.... in inference we are send boxes_transcripts folder which has .tsv files it contains bbox coordinates , transcripts along with what entity itself....then we got output know (its entity and text)...then what is the purpose of model.. ?

    Please guide me in this

    opened by NaveenVinayakS 0
  • Export trained model to ONNX

    Export trained model to ONNX

    How to export the Pick-Pytorch model into ONNX. I try but get an error because the keyword argument is used in forward(). If anyone converted the model into ONNX, please give some suggestions it will very helpful to me. Thank!

    opened by RahulJha11 0
  • Pre-training for the encoders

    Pre-training for the encoders

    I am training PICK on a privately annotated invoice dataset. At the moment LayoutLM V2's performance seems to be better than PICK. Will pre-training the encoders help? I am seeing that the resnet50 being used in pick has been modified. Can somebody suggest what would be the best method to boost a bit of performance? At the moment my training samples are less than 1000. Almost similar to SROIE.

    opened by AbhishekBose 0
  • Word Level Sequence Tagging

    Word Level Sequence Tagging

    Hi, Great work and tutorial to train custom KIE models. I have been training this model for ID cards and using iob_tagging_type as 'box_level'. When I read the paper I got to know it does sequence tagging of entities at character level which makes it little bit difficult to predict per box word level predictions. Giving example here:

    if there's a company_name: ASTER with bbox, the entity will get tagged with A,S,T,E,R.

    Suppose if decoder makes any mistake then the prediction for the company_name can be: ASTE (R did not predicted as the same entity)

    What to do in this type of scenarios? How we can customize decoded to predict word level predictions?

    opened by KananVyas 0
  • IndexError: Caught IndexError in DataLoader worker process 2

    IndexError: Caught IndexError in DataLoader worker process 2

    [2022-07-05 00:40:55,618 - train - INFO] - One GPU or CPU training mode start... [2022-07-05 00:40:56,108 - train - INFO] - Dataloader instances created. Train datasets: 1850 samples Validation datasets: 1850 samples. [2022-07-05 00:40:57,098 - train - INFO] - Model created, trainable parameters: 68567386. [2022-07-05 00:40:57,099 - train - INFO] - Optimizer and lr_scheduler created. [2022-07-05 00:40:57,100 - train - INFO] - Max_epochs: 100 Log_per_step: 10 Validation_per_step: 50. [2022-07-05 00:40:57,100 - train - INFO] - Training start... [2022-07-05 00:40:57,173 - trainer - WARNING] - Training is using GPU 0! Traceback (most recent call last): File "train.py", line 162, in entry_point(config) File "train.py", line 126, in entry_point main(config, local_master, logger if local_master else None) File "train.py", line 74, in main trainer.train() File "/content/gdrive/MyDrive/hdr/PICK-pytorch/trainer/trainer.py", line 135, in train result_dict = self._train_epoch(epoch) File "/content/gdrive/MyDrive/hdr/PICK-pytorch/trainer/trainer.py", line 199, in _train_epoch for step_idx, input_data_item in enumerate(self.data_loader): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 345, in next data = self._next_data() File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 838, in _next_data return self._process_data(data) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 881, in _process_data data.reraise() File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in DataLoader worker process 2. Original Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/content/gdrive/MyDrive/hdr/PICK-pytorch/data_utils/pick_dataset.py", line 111, in getitem boxes_and_transcripts_file = self.get_ann_file(Path(dataitem['file_name']).stem) File "/content/gdrive/MyDrive/hdr/PICK-pytorch/data_utils/pick_dataset.py", line 100, in get_ann_file filename = list(self.boxes_and_transcripts_folder.glob(f'**/{basename}.*'))[0] IndexError: list index out of range

    Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner self.run() File "/usr/lib/python3.7/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/pin_memory.py", line 25, in _pin_memory_loop r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL) File "/usr/lib/python3.7/multiprocessing/queues.py", line 113, in get return _ForkingPickler.loads(res) File "/usr/local/lib/python3.7/dist-packages/torch/multiprocessing/reductions.py", line 294, in rebuild_storage_fd fd = df.detach() File "/usr/lib/python3.7/multiprocessing/resource_sharer.py", line 58, in detach return reduction.recv_handle(conn) File "/usr/lib/python3.7/multiprocessing/reduction.py", line 185, in recv_handle return recvfds(s, 1)[0] File "/usr/lib/python3.7/multiprocessing/reduction.py", line 153, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_SPACE(bytes_size)) ConnectionResetError: [Errno 104] Connection reset by peer

    opened by aboah1994 2
Owner
Wenwen Yu
Ph.D. student at Huazhong University of Science and Technology
Wenwen Yu
Implementing Graph Convolutional Networks and Information Retrieval Mechanisms using pure Python and NumPy

Implementing Graph Convolutional Networks and Information Retrieval Mechanisms using pure Python and NumPy

Noah Getz 3 Jun 22, 2022
Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks

Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks (SDPoint) This repository contains the cod

Jason Kuen 17 Jul 4, 2022
This is the repository for the AAAI 21 paper [Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning].

CG3 This is the repository for the AAAI 21 paper [Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning]. R

null 12 Oct 28, 2022
This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

Maurice Weiler 59 Dec 9, 2022
Train robotic agents to learn pick and place with deep learning for vision-based manipulation in PyBullet.

Ravens is a collection of simulated tasks in PyBullet for learning vision-based robotic manipulation, with emphasis on pick and place. It features a Gym-like API with 10 tabletop rearrangement tasks, each with (i) a scripted oracle that provides expert demonstrations (for imitation learning), and (ii) reward functions that provide partial credit (for reinforcement learning).

Google Research 367 Jan 9, 2023
Code for technical report "An Improved Baseline for Sentence-level Relation Extraction".

RE_improved_baseline Code for technical report "An Improved Baseline for Sentence-level Relation Extraction". Requirements torch >= 1.8.1 transformers

Wenxuan Zhou 74 Nov 29, 2022
A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.

Object Pose Estimation Demo This tutorial will go through the steps necessary to perform pose estimation with a UR3 robotic arm in Unity. You’ll gain

Unity Technologies 187 Dec 24, 2022
It helps user to learn Pick-up lines and share if he has a better one

Pick-up-Lines-Generator(Open Source) It helps user to learn Pick-up lines Share and Add one or many to the DataBase Unique SQLite DataBase AI Undercon

knock_nott 0 May 4, 2022
Unofficial TensorFlow implementation of Protein Interface Prediction using Graph Convolutional Networks.

[TensorFlow] Protein Interface Prediction using Graph Convolutional Networks Unofficial TensorFlow implementation of Protein Interface Prediction usin

YeongHyeon Park 9 Oct 25, 2022
FAMIE is a comprehensive and efficient active learning (AL) toolkit for multilingual information extraction (IE)

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

null 18 Sep 1, 2022
The official PyTorch implementation of recent paper - SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training

This repository is the official PyTorch implementation of SAINT. Find the paper on arxiv SAINT: Improved Neural Networks for Tabular Data via Row Atte

Gowthami Somepalli 284 Dec 21, 2022
Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

STCN Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [a

Rex Cheng 456 Dec 12, 2022
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019).

ClusterGCN ⠀⠀ A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019). A

Benedek Rozemberczki 697 Dec 27, 2022
Danfeng Hong, Lianru Gao, Jing Yao, Bing Zhang, Antonio Plaza, Jocelyn Chanussot. Graph Convolutional Networks for Hyperspectral Image Classification, IEEE TGRS, 2021.

Graph Convolutional Networks for Hyperspectral Image Classification Danfeng Hong, Lianru Gao, Jing Yao, Bing Zhang, Antonio Plaza, Jocelyn Chanussot T

Danfeng Hong 154 Dec 13, 2022
Graph Convolutional Networks in PyTorch

Graph Convolutional Networks in PyTorch PyTorch implementation of Graph Convolutional Networks (GCNs) for semi-supervised classification [1]. For a hi

Thomas Kipf 4.5k Dec 31, 2022
MVGCN: a novel multi-view graph convolutional network (MVGCN) framework for link prediction in biomedical bipartite networks.

MVGCN MVGCN: a novel multi-view graph convolutional network (MVGCN) framework for link prediction in biomedical bipartite networks. Developer: Fu Hait

null 13 Dec 1, 2022
Official DGL implementation of "Rethinking High-order Graph Convolutional Networks"

SE Aggregation This is the implementation for Rethinking High-order Graph Convolutional Networks. Here we show the codes for citation networks as an e

Tianqi Zhang (张天启) 32 Jul 19, 2022
Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition in CVPR19

2s-AGCN Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition in CVPR19 Note PyTorch version should be 0.3! For PyTor

LShi 547 Dec 26, 2022