Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Neural Networks.

Last update: Dec 14, 2022

Overview

Dynamic-Graphs-Construction

Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Neural Networks.

We give an example to understand the dynamic graphs: we leverage pretrained ResNet18 (Top1 acc: 94.78%) on CIFAR-10 to extract feature representations. Then we set N = 50, k = 3 for dynamic graphs construction. The result is shown in the gif below: Note that nodes of the same color represent samples of the same class. The darker the color of the edge, the higher the similarity between the two corresponding nodes. The modularity curve of this dynamic graph:

A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

An Introduction to Deep Learning for the Physical Layer An usable PyTorch implementation of the noisy autoencoder infrastructure in the paper "An Intr

120 Nov 21, 2022

Comments

modularity计算出来为负值是什么原因？

您好！我用的自己的数据集在resnet18上用了您的方法做了模块化程度的计算，提取的是relu层之后的特征，发现计算每层modularity会出现负值，这是什么原因呢？之后又尝试提取bn层后的特征还是有大量的负值存在，请问这是正常现象么？

一下是几次计算的结果，len（ modularity ）=17层 modularity: [-0.019196462265709934, -0.02271897533277316, -0.023171874127450653, -0.022470118258014187, -0.01645792102456704, -0.018011648884517804, -0.010016131091073174, -0.02012415433281817, -0.00993257811055065, -0.008657139629071227, -0.010918394908571914, -0.009388655522269241, -0.002584169673622568, -0.0005555165491241826, -0.012236681334620234, -0.013657490194399866, -0.001615048060680923]

modularity: [-0.009842042437187018, -0.009372720350310716, -0.012085664778406252, 0.005141147857948121, 0.010048463577370794, 0.00018423969515977273, -0.0030635190699569922, -0.00026638565957969734, -0.010287994904722021, -0.012399507863719801, -0.008366178789636743, -0.0032961831139690576, -0.006384714434207794, 0.0067996706828107704, -0.005832453918450487, 0.009217809629457067, -0.006108037287940963]

[-0.008759199831022697, -0.036767948297815836, -0.029265369915441954, -0.025968150236407347, -0.025383781534253247, -0.019468812859621223, -0.012428653247791372, -0.0227319975256084, -0.028128953521547748, -0.02349360946111945, -0.024317913360711686, -0.02071382370153553, -0.02179457870885567, -0.015504702593439782, -0.0127327671915606, -0.005047404214200173, -0.01602515198867172]

opened by Z-Yh-June 3

Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Neural Networks.

Related tags

Overview

Dynamic-Graphs-Construction

You might also like...

[CIKM 2019] Code and dataset for "Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction"

Towards Part-Based Understanding of RGB-D Scans

Towards Long-Form Video Understanding

[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.

[CVPR2022] Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos

Pytorch implementation of "Forward Thinking: Building and Training Neural Networks One Layer at a Time"

OptNet: Differentiable Optimization as a Layer in Neural Networks

A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

Comments

modularity计算出来为负值是什么原因？

Owner

A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

Code for the CVPR 2021 paper: Understanding Failures of Deep Networks via Robust Feature Extraction

An Inverse Kinematics library aiming performance and modularity

Code for Understanding Pooling in Graph Neural Networks

Source code of NeurIPS 2021 Paper ''Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration''

Pytorch code for "State-only Imitation with Transition Dynamics Mismatch" (ICLR 2020)

TART - A PyTorch implementation for Transition Matrix Representation of Trees with Transposed Convolutions

Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021)

We have implemented shaDow-GNN as a general and powerful pipeline for graph representation learning. For more details, please find our paper titled Deep Graph Neural Networks with Shallow Subgraph Samplers, available on arXiv (https//arxiv.org/abs/2012.01380).