Code for the paper "How Attentive are Graph Attention Networks?"

Last update: Dec 29, 2022

Related tags

Deep Learning how_attentive_are_gats

Overview

How Attentive are Graph Attention Networks?

This repository is the official implementation of How Attentive are Graph Attention Networks?.

The PyTorch Geometric implementation of the new GATv2 layer is in this main directory.

The rest of the code for reproducing the experiments in the paper will be made publicly available.

Comments

Publishing code for reproduction

Thank you for your great contribution! Currently, we are doing research with GAT, GAT-V2 (and Graph Transformers) and try to reproduce your results on the ogbn-datasets. Do you already know when you will publish the rest of the code?

Best regards

opened by LeadBeetle 5
Reproducing results on proteins

Hi!

I was trying to reproduce the results on the proteins dataset on a V100 GPU, and I am running into a few problems. First I had some issues with the BatchSampler (samplers cannot be passed to iterative datasets), so I just removed it (I am using the latest version of dgl, since 0.6 is not available with my CUDA version).

After fixing that, I came across this error message ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 64]) in line 98 of models.py https://github.com/tech-srl/how_attentive_are_gats/blob/b5ccb6129878bb6f6f8c3f77934046d8e0920a52/proteins/models.py#L90-L100

Is this error related with removing the BatchSampler? And which tensor size is it expected at that line?

Thanks for the help

opened by adrianjav 4
(Possible) bug when graph.is_block is True

Sorry for bothering again.

This might be dumb, but I am reading the DGL implementation to understand it, and I really don't get L272.

https://github.com/tech-srl/how_attentive_are_gats/blob/b5ccb6129878bb6f6f8c3f77934046d8e0920a52/gatv2_conv_DGL.py#L266-L272

The only thing that makes sense to me is that the rhs was supposed to be feat_dst as well, and this is some bug. Otherwise, it does not make sense to set share_weights=False when graph.is_block is True. Am I missing something?

opened by adrianjav 3
Number of features does not match the number of nodes when num_heads > 1

Hi, I am using the DGL based code of the project. When I set num_heads = 2 to initialize the GATv2Conv layers I receive the following error

dgl._ffi.base.DGLError: Expect number of features to match number of nodes (len(u)). Got 568 and 284 instead

I guess the attention heads are not being aggregated? I noticed the DGL version does not have an is_concat option like in the annotated code demo

opened by Jonbroad15 2
Reproduce the GAT v1 attention matrix

Thanks for your great contribution!! I'm confused about Figure 1 (a) in your paper. Which layer of GAT is this attention matrix in? Is the attention matrix of all layers the same? Is the attention matrix between different heads in one layer like this?

Best regards

opened by ALEX13679173326 3

Owner

GitHub

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow

18 Oct 6, 2022

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

66 Dec 26, 2022

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

29 Dec 28, 2022

TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

73 Nov 6, 2022

Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

1 Oct 26, 2021

Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

This codebase is being actively maintained, please create and issue if you have issues using it Basics All data files are included under losses and ea

32 Nov 9, 2021

Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

138 Dec 12, 2022

Code for ACM MM 2020 paper "NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination"

NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination The offical implementation for the "NOH-NMS: Improving Pedestrian Detection by

64 Nov 11, 2022

Official TensorFlow code for the forthcoming paper

~ Efficient-CapsNet ~ Are you tired of over inflated and overused convolutional neural networks? You're right! It's time for CAPSULES :)

203 Jan 8, 2023

This is the code for the paper "Contrastive Clustering" (AAAI 2021)

Contrastive Clustering (CC) This is the code for the paper "Contrastive Clustering" (AAAI 2021) Dependency python>=3.7 pytorch>=1.6.0 torchvision>=0.8

210 Dec 30, 2022

Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

Computer Vision Lab at Columbia University

139 Nov 18, 2022

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning This is the PyTorch implementation of our paper: FeatMatch: Feature-Based Augmentat

43 Nov 19, 2022

Code for the paper A Theoretical Analysis of the Repetition Problem in Text Generation

A Theoretical Analysis of the Repetition Problem in Text Generation This repository share the code for the paper "A Theoretical Analysis of the Repeti

37 Nov 21, 2022

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

199 Jan 8, 2023

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

296 Dec 29, 2022