Graph Representation Learning via Graphical Mutual Information Maximization

Related tags

Deep Learning GMI
Overview

GMI (Graphical Mutual Information)

Graph Representation Learning via Graphical Mutual Information Maximization (Peng Z, Huang W, Luo M, et al., WWW 2020): https://arxiv.org/abs/2002.01169

image

Overview

Note that we propose two variants of GMI in the paper, the one is GMI-mean, and the other is GMI-adaptive. Since GMI-mean often outperforms GMI-adaptive (see the experiments in the paper), here we give a PyTorch implementation of GMI-mean. To make GMI more practical, we provide an alternative solution to compute FMI. Such a solution still ensures the effectiveness of GMI and improves the efficiency greatly. The repository is organized as follows:

  • data/ includes three benchmark datasets;
  • models/ contains the implementation of the GMI pipeline (gmi.py) and the logistic regression classifier (logreg.py);
  • layers/ contains the implementation of a standard GCN layer (gcn.py), the bilinear discriminator (discriminator.py), and the mean-pooling operator (avgneighbor.py);
  • utils/ contains the necessary processing tool (process.py).

To better understand the code, we recommend that you could read the code of DGI/Petar (https://arxiv.org/abs/1809.10341) in advance. Besides, you could further optimize the code based on your own needs. We display it in an easy-to-read form.

Requirements

  • PyTorch 1.2.0
  • Python 3.6

Usage

python execute.py

Cite

Please cite our paper if you make advantage of GMI in your research:

@inproceedings{
peng2020graph,
title="{Graph Representation Learning via Graphical Mutual Information Maximization}",
author={Peng, Zhen and Huang, Wenbing and Luo, Minnan and Zheng, Qinghua and Rong, Yu and Xu, Tingyang and Huang, Junzhou},
booktitle={Proceedings of The Web Conference},
year={2020},
doi={https://doi.org/10.1145/3366423.3380112},
}
You might also like...
An official PyTorch implementation of the TKDE paper "Self-Supervised Graph Representation Learning via Topology Transformations".

Self-Supervised Graph Representation Learning via Topology Transformations This repository is the official PyTorch implementation of the following pap

Hsiang Gao 2 Oct 31, 2022
Code release for The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification (TIP 2020)
Code release for The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification (TIP 2020)

The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification Code release for The Devil is in the Channels: Mutual-Channel

PRIS-CV: Computer Vision Group 230 Dec 31, 2022
Official PyTorch code for Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)
Official PyTorch code for Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)

Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021) This repository is the official PyTorc

Jingyun Liang 139 Dec 29, 2022
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations

Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code repo for paper Trans-Encoder: Unsupervised sentence-pa

Amazon 101 Dec 29, 2022
Self-supervised learning on Graph Representation Learning (node-level task)

graph_SSL Self-supervised learning on Graph Representation Learning (node-level task) How to run the code To run GRACE, sh run_GRACE.sh To run GCA, sh

Namkyeong Lee 3 Dec 31, 2021
pytorch implementation of
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
Eff video representation - Efficient video representation through neural fields

Neural Residual Flow Fields for Efficient Video Representations 1. Download MPI

null 41 Jan 6, 2023
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).
This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).

Predicting Patient Outcomes with Graph Representation Learning This repository contains the code used for Predicting Patient Outcomes with Graph Repre

Emma Rocheteau 76 Dec 22, 2022
Comments
  • How to get the reported accuracy on citation dataset？

    How to get the reported accuracy on citation dataset？

    Thank you for sharing your code！ I didn't modify any code, just ran the code, but the accuracy is lower than that reported on three citation datasets in the paper. Should some hyper-parameters be tuned?

    opened by wtzhao1631 0
  • About link prediction task

    About link prediction task

    Hi. I'm really impressed to your works in that it has huge possibility to be extended.

    I have a simple question. In 'execute.py', there is a LogReg process to regress the embedding from your model. Did you also use LogReg to predict links? Or did you use a simple dot product decoder like GAEs?

    opened by SeongJinAhn 1
  • On calculating FMI in code

    On calculating FMI in code

    Hi and thank you for sharing your great work. In the code for calculating FMI you use two discriminators, one for discriminating GCN output and input features, which calculates mutual information between hi and xi for all nodes, and another for discriminating avrage_neighbor output and GCN output which calculates mutual information between hi and xj ( j is a neighbor of i). xj is calculated via average_neighbor by averaging the compressed input feature, XW(0). I have some questions about this: Why do you use prelu after averaging features of the node's neighbors? Why do you compress input for the second discriminator and not for the first one? When combining the output of discriminators you use different parameters for them, alpha and betha. Why?

    opened by shirinmous 2
  • Effectiveness of Objective Function

    Effectiveness of Objective Function

    hi, thanks for your perfect job, and I like it very much! However, I have a small question. In the section 4.4 of origin paper, you test the effectiveness of objective function, and you say that you replace your objective function with DGI loss., for example. So, what's the difference between this operation and run DGI source code directly? In my opinion, use the DGI loss and don't maximize GMI, which equals to run source code...Maybe, I am wrong. Could you please tell me the difference? I wish your reply. Best wishes!

    opened by liun-online 0
Owner
null
GitHub
Joint learning of images and text via maximization of mutual information

mutual_info_img_txt Joint learning of images and text via maximization of mutual information. This repository incorporates the algorithms presented in

Ruizhi Liao 10 Dec 22, 2022
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Deep Cognition and Language Research (DeCLaRe) Lab 89 Dec 26, 2022
Python Implementation of algorithms in Graph Mining, e.g., Recommendation, Collaborative Filtering, Community Detection, Spectral Clustering, Modularity Maximization, co-authorship networks.

Graph Mining Author: Jiayi Chen Time: April 2021 Implemented Algorithms: Network: Scrabing Data, Network Construbtion and Network Measurement (e.g., P

Jiayi Chen 3 Mar 3, 2022
(AAAI2020)Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing

Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing This repository contains pytorch source code for AAAI2020 oral paper: Grapy-ML

null 54 Aug 4, 2022
Self-Supervised Learning with Kernel Dependence Maximization

Self-Supervised Learning with Kernel Dependence Maximization This is the code for SSL-HSIC, a self-supervised learning loss proposed in the paper Self

DeepMind 29 Dec 29, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 29 Dec 28, 2022
RE3: State Entropy Maximization with Random Encoders for Efficient Exploration

State Entropy Maximization with Random Encoders for Efficient Exploration (RE3) (ICML 2021) Code for State Entropy Maximization with Random Encoders f

Younggyo Seo 47 Nov 29, 2022
Code, final versions, and information on the Sparkfun Graphical Datasheets

Graphical Datasheets Code, final versions, and information on the SparkFun Graphical Datasheets. Generated Cells After Running Script Example Complete

SparkFun Electronics 102 Jan 5, 2023
We have implemented shaDow-GNN as a general and powerful pipeline for graph representation learning. For more details, please find our paper titled Deep Graph Neural Networks with Shallow Subgraph Samplers, available on arXiv (https//arxiv.org/abs/2012.01380).

Deep GNN, Shallow Sampling Hanqing Zeng, Muhan Zhang, Yinglong Xia, Ajitesh Srivastava, Andrey Malevich, Rajgopal Kannan, Viktor Prasanna, Long Jin, R

Facebook Research 117 Dec 20, 2022
Some tentative models that incorporate label propagation to graph neural networks for graph representation learning in nodes, links or graphs.

Some tentative models that incorporate label propagation to graph neural networks for graph representation learning in nodes, links or graphs.

zshicode 1 Nov 18, 2021