Contrastive Multi-View Representation Learning on Graphs

Related tags

Deep Learning mvgrl
Overview

Contrastive Multi-View Representation Learning on Graphs

This work introduces a self-supervised approach based on contrastive multi-view learning to learn node and graph level representations.

It has been accepted at ICML 2020:

https://arxiv.org/abs/2006.05582



Reference

@incollection{icml2020_1971,
 author = {Hassani, Kaveh and Khasahmadi, Amir Hosein},
 booktitle = {Proceedings of International Conference on Machine Learning},
 pages = {3451--3461},
 title = {Contrastive Multi-View Representation Learning on Graphs},
 year = {2020}
}
Comments
  • The split of Cora is different from the original setting

    The split of Cora is different from the original setting

    Dear authors, I have noticed that you use DGL to load Cora dataset. But the corresponding data split is not the correct split used in GCN paper. And someone has already pointed out this issue (https://github.com/rusty1s/pytorch_geometric/issues/891). Thus your comparison with baselines on Cora dataset is invalid.

    opened by wzfhaha 2
  • Question about the reported results.

    Question about the reported results.

    Hi @kavehhassani ,

    Thanks for the released code! I have a minor question about the reported results. When I ran the code, some results are printed and I wonder how to obtain the reported results in the paper (e.g. 89.7 for MUTAG dataset in graph classification). Is it maybe just to pick out the highest result? Thanks a lot!

    Here are some of results printed: ####################MUTAG#################### Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 20, Seed: 123 0.8880701754385966 0.0732108857685375 Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 20, Seed: 132 0.8919590643274855 0.07107527976346202 Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 20, Seed: 321 0.898684210526316 0.06647971246853003 Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 20, Seed: 312 0.8581578947368422 0.07014429742760363 Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 20, Seed: 231 0.8964327485380117 0.08969212337006427 Dataset: MUTAG, Layer:2, Batch: 32, Epoch: 40, Seed: 123 0.8545029239766082 0.08952242299027656

    opened by ha-lins 2
  • Bump urllib3 from 1.25.6 to 1.25.8

    Bump urllib3 from 1.25.6 to 1.25.8

    Bumps urllib3 from 1.25.6 to 1.25.8.

    Release notes

    Sourced from urllib3's releases.

    1.25.8

    Release: 1.25.8

    1.25.7

    No release notes provided.

    Changelog

    Sourced from urllib3's changelog.

    1.25.8 (2020-01-20)

    • Drop support for EOL Python 3.4 (Pull #1774)

    • Optimize _encode_invalid_chars (Pull #1787)

    1.25.7 (2019-11-11)

    • Preserve chunked parameter on retries (Pull #1715, Pull #1734)

    • Allow unset SERVER_SOFTWARE in App Engine (Pull #1704, Issue #1470)

    • Fix issue where URL fragment was sent within the request target. (Pull #1732)

    • Fix issue where an empty query section in a URL would fail to parse. (Pull #1732)

    • Remove TLS 1.3 support in SecureTransport due to Apple removing support (Pull #1703)

    Commits
    • 2a57bc5 Release 1.25.8 (#1788)
    • a2697e7 Optimize _encode_invalid_chars (#1787)
    • d2a5a59 Move IPv6 test skips in server fixtures
    • d44f0e5 Factorize test certificates serialization
    • 84abc7f Generate IPV6 certificates using trustme
    • 6a15b18 Run IPv6 Tornado server from fixture
    • 4903840 Use trustme to generate IP_SAN cert
    • 9971e27 Empty responses should have no lines.
    • 62ef68e Use trustme to generate NO_SAN certs
    • fd2666e Use fixture to configure NO_SAN test certs
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • S3 bucket for data file download doesn't exist

    S3 bucket for data file download doesn't exist

    Use the download function in dataset.py wiil raise error,open the "https://s3.us-east-2.amazonaws.com/dgl.ai/dataset/cora_raw.zip" in browser directly shows xml like below

    <Error>
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist</Message>
<BucketName>dgl.ai</BucketName>
<RequestId>RVE9W568FFMAPKQG</RequestId>
<HostId>+Ud8WlXz5mM1uYadhxlJzwnmoFph4FHjAEwvpvxl5z9fUUSlN2qJ5Z9w3nnsBEcseCs9J5XWJR0=</HostId>
</Error>
    opened by GalliumWang 1
  • Efficience on large graph

    Efficience on large graph

    I'm interested in self-supervised learning on the graph and mvgrl is an excellent work that combining self-supervised learning with multi-view augmentation. However, I'm also curious about its usage in large graphs such as knowledge graph. May I ask you about the suggestion or efficient training method on the augmentation operation on the knowledge graph? Because in the way of mvgrl , it needs 4 GNN encoders at the same time per epoch. I try to experiment with it but it's extremely time-consuming. Thanks a lot in advance~

    opened by junkangwu 1
  • A small problem in the code

    A small problem in the code

    Dear authors, I notice that the code "adj = normalize_adj(adj + sp.eye(adj.shape[0])).todense()" at line 64 of the path "node/dataset.py" has already added the self-loop. But it adds the self-loop again at line 62 of the path "utils.py".

    opened by Andrewsama 1
  • “Kill” problem when I ran the REDDIT-BINARY and REDDIT-MULTI datasets

    “Kill” problem when I ran the REDDIT-BINARY and REDDIT-MULTI datasets

    Hi @kavehhassani,

    Thanks for the nice work and your released code.

    I met a "kill" problem by the os when I loaded these two big datasets. It is caused by the huge memory usage (100G+) of this for loop. Could you give me some advice?

    Thanks in advance!

    opened by ha-lins 1
  • Bump numpy from 1.17.4 to 1.22.0

    Bump numpy from 1.17.4 to 1.22.0

    Bumps numpy from 1.17.4 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Is this an Error about the code in graph/train.py？

    Is this an Error about the code in graph/train.py？

    when I run and debug the code in graph/train.py

    Use the MUTAG dataset for example:

    258 np.random.shuffle(train_idx) this line shuffled the 188 graph index 265 mask = num_nodes[idx: idx + batch_size] , but the line 265 to read the num_nodes as a mask, still use the original index?

    And another question:

    Is the mask in 265 necessaries， when in the GCN class, the forward function does not use the mask for calculate? def forward(self, feat, adj, mask):

    opened by xdjwolf 0
  • inconsistency in requirements.txt

    inconsistency in requirements.txt

    dgl==0.4.1 and torch==1.3.1 are not available when I try to install the requirements. While latest dgl seems not working. Can you please upgrade the requirements.txt with relevant and available versions?

    image

    opened by mhadnanali 0
  • Code confusion

    Code confusion

    Hi: I'm highly confused that diffusion shown in paper, in code I only see two graph adjacency matrix employed to same instance, so how to show diffusion?

    opened by wangzeyu135798 0
  • Bump urllib3 from 1.25.6 to 1.26.5

    Bump urllib3 from 1.25.6 to 1.26.5

    Bumps urllib3 from 1.25.6 to 1.26.5.

    Release notes

    Sourced from urllib3's releases.

    1.26.5

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.4

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.3

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed bytes and string comparison issue with headers (Pull #2141)

    • Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme (Pull #2107)

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.2

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

    1.26.1

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

    1.26.0

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

    • Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning should opt-in explicitly by setting ssl_version=ssl.PROTOCOL_TLSv1_1 (Pull #2002) Starting in urllib3 v2.0: Connections that receive a DeprecationWarning will fail

    • Deprecated Retry options Retry.DEFAULT_METHOD_WHITELIST, Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST and Retry(method_whitelist=...) in favor of Retry.DEFAULT_ALLOWED_METHODS, Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT, and Retry(allowed_methods=...) (Pull #2000) Starting in urllib3 v2.0: Deprecated options will be removed

    ... (truncated)

    Changelog

    Sourced from urllib3's changelog.

    1.26.5 (2021-05-26)

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    1.26.4 (2021-03-15)

    • Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

    1.26.3 (2021-01-26)

    • Fixed bytes and string comparison issue with headers (Pull #2141)

    • Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme. (Pull #2107)

    1.26.2 (2020-11-12)

    • Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

    1.26.1 (2020-11-11)

    • Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

    1.26.0 (2020-11-10)

    • NOTE: urllib3 v2.0 will drop support for Python 2. Read more in the v2.0 Roadmap <https://urllib3.readthedocs.io/en/latest/v2-roadmap.html>_.

    • Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

    • Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning

    ... (truncated)

    Commits
    • d161647 Release 1.26.5
    • 2d4a3fe Improve performance of sub-authority splitting in URL
    • 2698537 Update vendored six to 1.16.0
    • 07bed79 Fix deprecation warnings for Python 3.10 ssl module
    • d725a9b Add Python 3.10 to GitHub Actions
    • 339ad34 Use pytest==6.2.4 on Python 3.10+
    • f271c9c Apply latest Black formatting
    • 1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
    • a891304 Release 1.26.4
    • 8d65ea1 Merge pull request from GHSA-5phf-pp7p-vc2r
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Owner
Kaveh
Principal AI Research Scientist
Kaveh
GitHub
Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Blender add-on: Camera additions In 3D view, it adds these actions to the View|Cameras menu: View → Camera : set the current camera to the 3D view Vie

German Bauer 11 Feb 8, 2022
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
PyTorch implementation for COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (CVPR 2021)

Completer: Incomplete Multi-view Clustering via Contrastive Prediction This repo contains the code and data of the following paper accepted by CVPR 20

XLearning Group 72 Dec 7, 2022
ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representa

Bats Research 94 Nov 21, 2022
Some tentative models that incorporate label propagation to graph neural networks for graph representation learning in nodes, links or graphs.

Some tentative models that incorporate label propagation to graph neural networks for graph representation learning in nodes, links or graphs.

zshicode 1 Nov 18, 2021
Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training This project hosts the code for implementing the DenseCL algorithm for se

Xinlong Wang 491 Jan 3, 2023
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning This repository contains the code and relevant instructions

XiaoMing 5 Aug 19, 2022
SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING" i

Saeed Lotfi 28 Dec 12, 2022
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo This repository includes the source code for our CVPR 2021 paper on multi-view mult

Jiahao Lin 66 Jan 4, 2023
PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

How to Reproduce our Results This repository contains PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Represen

opcrisis 46 Dec 15, 2022
Code in conjunction with the publication 'Contrastive Representation Learning for Hand Shape Estimation'

HanCo Dataset & Contrastive Representation Learning for Hand Shape Estimation Code in conjunction with the publication: Contrastive Representation Lea

Computer Vision Group, Albert-Ludwigs-Universität Freiburg 38 Dec 13, 2022
Object-aware Contrastive Learning for Debiased Scene Representation

Object-aware Contrastive Learning Official PyTorch implementation of "Object-aware Contrastive Learning for Debiased Scene Representation" by Sangwoo

null 43 Dec 14, 2022
Object-aware Contrastive Learning for Debiased Scene Representation

Object-aware Contrastive Learning Official PyTorch implementation of "Object-aware Contrastive Learning for Debiased Scene Representation" by Sangwoo

null 43 Dec 14, 2022
Pytorch Implementation of "Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation"

CRL_EGPG Pytorch Implementation of Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation We use contrastive loss implemented b

YHR 25 Nov 14, 2022
[CVPR 2022 Oral] Rethinking Minimal Sufficient Representation in Contrastive Learning

Rethinking Minimal Sufficient Representation in Contrastive Learning PyTorch implementation of Rethinking Minimal Sufficient Representation in Contras

null 36 Nov 23, 2022
[CVPR 2022 Oral] Crafting Better Contrastive Views for Siamese Representation Learning

Crafting Better Contrastive Views for Siamese Representation Learning (CVPR 2022 Oral) 2022-03-29: The paper was selected as a CVPR 2022 Oral paper! 2

null 249 Dec 28, 2022
Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021)

Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021) Authors: Xinshi Chen, Haoran Sun, Caleb Ellington, Eric Xing, Le Song Link to pap

Xinshi Chen 2 Dec 20, 2021