Continuous Diffusion Graph Neural Network

Hello,

When I run python run_GNN.py --dataset Cora , it creates the following error.

Traceback (most recent call last): File "run_GNN.py", line 401, in main(opt) File "run_GNN.py", line 190, in main dataset = get_dataset(opt, '../data', opt['not_lcc']) File "/home/saydemr/Documents/graph-neural-pde/src/data.py", line 36, in get_dataset dataset = Planetoid(path, ds) File "/home/saydemr/anaconda3/envs/grand/lib/python3.7/site-packages/torch_geometric/datasets/planetoid.py", line 56, in init self.data, self.slices = torch.load(self.processed_paths[0]) File "/home/saydemr/anaconda3/envs/grand/lib/python3.7/site-packages/torch/serialization.py", line 592, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/home/saydemr/anaconda3/envs/grand/lib/python3.7/site-packages/torch/serialization.py", line 851, in _load result = unpickler.load() ModuleNotFoundError: No module named 'torch_geometric.data.storage'

System : Ubuntu 20.04 Torch : 1.8.1+cu102 Cuda : release 10.2, V10.2.89 torch-geometric==1.7.0

On torch-geometric==2.0.2 however, this error turns into the following

Traceback (most recent call last): File "run_GNN.py", line 401, in main(opt) File "run_GNN.py", line 202, in main model = GNN(opt, dataset, device).to(device) if opt["no_early"] else GNNEarly(opt, dataset, device).to(device) File "/home/saydemr/Documents/graph-neural-pde/src/GNN_early.py", line 25, in init self.odeblock = block(self.f, self.regularization_fns, opt, dataset.data, device, t=time_tensor).to(device) File "/home/saydemr/Documents/graph-neural-pde/src/block_transformer_attention.py", line 16, in init dtype=data.x.dtype) File "/home/saydemr/Documents/graph-neural-pde/src/utils.py", line 113, in get_rw_adj edge_index, edge_weight, fill_value, num_nodes) File "/home/saydemr/anaconda3/envs/grand/lib/python3.7/site-packages/torch_geometric/utils/loop.py", line 220, in add_remaining_self_loops raise AttributeError("No valid 'fill_value' provided") AttributeError: No valid 'fill_value' provided

Hello when I run python run_python run_image.py I have this error

[pyKeOps]: Warning, cuda was detected, but driver API could not be initialized. Switching to cpu only. Loading Data creating in_memory_datasets creating GNN model Traceback (most recent call last): File "run_image.py", line 349, in main(opt) File "run_image.py", line 183, in main model = GNN_image(opt, batch, opt['num_class'], device).to(device) File "/home/boudjelala/Desktop/Tests_code/Abder_GNN/graph-neural-pde/src/GNN_image.py", line 20, in init super(GNN_image, self).init(opt, data, num_classes, device) TypeError: init() takes from 3 to 4 positional arguments but 5 were given

Thanks

Hello,

Running the GNN on newer GPU's raise the following warning.

Then, after a while the program quits with the following error.

I understand that these warnings and errors are related with Nvidia and/or PyTorch. However, are there any options that you can update the environment dependencies with newer versions of PyTorch? Maybe newer versions of other packages as well?

Thanks!

Hello when I run python run_GNN.py --dataset Cora I have this error File "/home/GNN/graph-neural-pde/src/run_GNN.py", line 294, in main(opt) File "/home/GNN/graph-neural-pde/src/run_GNN.py", line 161, in main tmp_train_acc, tmp_val_acc, tmp_test_acc = this_test(model, data, opt) File "/home/anaconda3/envs/myenv/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "/home/GNN/graph-neural-pde/src/run_GNN.py", line 94, in test logits, accs = model(feat), [] File "/home/anaconda3/envs/myenv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/GNN/graph-neural-pde/src/GNN_early.py", line 70, in forward z = self.odeblock(x) File "/home/anaconda3/envs/myenv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/GNN/graph-neural-pde/src/block_transformer_attention.py", line 59, in forward state_dt = integrator( File "/home/GNN/graph-neural-pde/src/early_stop_solver.py", line 277, in call shapes, func, y0, t, rtol, atol, method, options = _check_inputs(func, y0, self.t, rtol, atol, method, options, SOLVERS) NameError: name 'SOLVERS' is not defined

First of all great paper and thanks for releasing the code, just a couple of minor fixes in the visualise_attention.ipynb

1. Wrong module import at the top of the notebook. run_CGNN has been used, instead run_GNN should be used instead.
2. In the data.py script, the get_dataset function requires opt to be a dict, viz https://github.com/twitter-research/graph-neural-pde/blob/4323db3bb3badbcfc3c569635b7f8f072946528d/src/data.py#L29 but the dataset name has been passed as a str. Changing the line to dataset = dt.get_dataset({'dataset':'Cora', 'rewiring' : None}, '/content/graph-neural-pde/data', use_lcc=True) for simplicity, or adding a dataset key to opt would fix the issue.
3. A couple of values are missing in the opt such as use_mlp, use_labels, fc_out, label_rate,batch_norm, reweight_attention, attention_dim,label_rate, reweight_attention, step_size, max_nfe, no_alpha_sigmoid, and add_source. Maybe change the opt to the one from test_params.py or adding the aforementioned params would fix the issue.
4. Maybe add a Colab Notebook. I've been able to successfully reproduce the notebook in this gist, I'd love to open up PR adding a Colab.

Dear authors,

I have read your works on GRAND and BLEND papers and have checked the implementation for those papers.

I have a following question. In src/best_params.py I saw that

'Cora': {'M_nodes': 64, 'adaptive': False, 'add_source': True, 'adjoint': False, 'adjoint_method': 'adaptive_heun', 'adjoint_step_size': 1, 'alpha': 1.0, 'alpha_dim': 'sc', 'att_samp_pct': 1, 'attention_dim': 128, 'attention_norm_idx': 1, 'attention_rewiring': False, 'attention_type': 'scaled_dot', 'augment': False, 'baseline': False, 'batch_norm': False, 'beltrami': False, 'beta_dim': 'sc', 'block': 'attention', 'cpus': 1, 'data_norm': 'rw', 'dataset': 'Cora', 'decay': 0.00507685443154266, 'directional_penalty': None, 'dropout': 0.046878964627763316, 'dt': 0.001, 'dt_min': 1e-05, 'epoch': 100, 'exact': True, 'fc_out': False, 'feat_hidden_dim': 64, 'function': 'laplacian', 'gdc_avg_degree': 64, 'gdc_k': 64, 'gdc_method': 'ppr', 'gdc_sparsification': 'topk', 'gdc_threshold': 0.01, 'gpus': 0.5, 'grace_period': 20, 'heads': 8, 'heat_time': 3.0, 'hidden_dim': 80, 'input_dropout': 0.5, 'jacobian_norm2': None, 'kinetic_energy': None, 'label_rate': 0.5, 'leaky_relu_slope': 0.2, 'lr': 0.022924849756740397, 'max_epochs': 1000, 'max_iters': 100, 'max_nfe': 2000, 'method': 'dopri5', 'metric': 'accuracy', 'mix_features': False, 'name': 'cora_beltrami_splits', 'new_edges': 'random', 'no_alpha_sigmoid': False, 'not_lcc': True, 'num_init': 1, 'num_samples': 1000, 'num_splits': 2, 'ode_blocks': 1, 'optimizer': 'adamax', 'patience': 100, 'pos_enc_hidden_dim': 16, 'pos_enc_orientation': 'row', 'pos_enc_type': 'GDC', 'ppr_alpha': 0.05, 'reduction_factor': 10, 'regularise': False, 'reweight_attention': False, 'rewire_KNN': False, 'rewire_KNN_T': 'T0', 'rewire_KNN_epoch': 10, 'rewire_KNN_k': 64, 'rewire_KNN_sym': False, 'rewiring': None, 'rw_addD': 0.02, 'rw_rmvR': 0.02, 'self_loop_weight': 1, 'sparsify': 'S_hat', 'square_plus': True, 'threshold_type': 'addD_rvR', 'time': 18.294754260552843, 'tol_scale': 821.9773048827274, 'tol_scale_adjoint': 1.0, 'total_deriv': None, 'use_cora_defaults': False, 'use_flux': False, 'use_labels': False, 'use_lcc': True, 'use_mlp': False},

I notice that there are several parameters was used here came from some unknown process, such as:

• 'decay': 0.00507685443154266
• 'dropout': 0.046878964627763316
• 'tol_scale': 821.9773048827274

So how could you get those above hyperparameter. I have asked my senior, he said that maybe it come from some Bayesian Optimization based on Gaussian Process. But we are not sure about our idea.

So can you explain for me, which methods you have used and how can it be implement.

Thanks in advance,

Best regards, Tien Dung

Hi,

could you explain how you generate the setting for table 2 in the paper?

in the paper, you state that " for all datasets using 100 random splits with 20 random initializations." which means that the set_train_val_test_split function should generate these splits. could you elaborate on how these parts work in your code?

Hi there, I have installed the packages as stated in README.md and tried to run script run_GNN.py. It turns of I got a RuntimeError with the error message as follows:

Traceback (most recent call last): File "run_GNN.py", line 7, in from torch_geometric.nn import GCNConv, ChebConv # noqa File "/usr/local/lib/python3.7/dist-packages/torch_geometric/init.py", line 4, in import torch_geometric.data File "/usr/local/lib/python3.7/dist-packages/torch_geometric/data/init.py", line 1, in from .data import Data File "/usr/local/lib/python3.7/dist-packages/torch_geometric/data/data.py", line 20, in from torch_sparse import SparseTensor File "/usr/local/lib/python3.7/dist-packages/torch_sparse/init.py", line 41, in from .tensor import SparseTensor # noqa File "/usr/local/lib/python3.7/dist-packages/torch_sparse/tensor.py", line 13, in class SparseTensor(object): File "/usr/local/lib/python3.7/dist-packages/torch/jit/_script.py", line 974, in script _compile_and_register_class(obj, _rcb, qualified_name) File "/usr/local/lib/python3.7/dist-packages/torch/jit/_script.py", line 67, in _compile_and_register_class torch._C._jit_script_class_compile(qualified_name, ast, defaults, rcb) RuntimeError: Tried to access nonexistent attribute or method 'crow_indices' of type 'Tensor'.: File "/usr/local/lib/python3.7/dist-packages/torch_sparse/tensor.py", line 109 def from_torch_sparse_csr_tensor(self, mat: torch.Tensor, has_value: bool = True): rowptr = mat.crow_indices() ~~~~~~~~~~~~~~~~ <--- HERE col = mat.col_indices()

Can anyone explain it and maybe try to help me resolve this? Thank you

Best regards, Simon

I was running the data_image.py file, with the code in the last lines uncommented hoping it will produce an image similar to figure 4 in the GRAND paper. However, I am missing the SuperPixel dataset as well as the SuperPixelData module to import the functions. From which repository can I clone them?

I want to test run_image.py function but not working, please give me a solution Thanks

/graph-neural-pde/src$ python run_image.py [pyKeOps]: Warning, cuda was detected, but driver API could not be initialized. Switching to cpu only. Loading Data creating in_memory_datasets $anaconda3/envs/grand_2/lib/python3.7/site-packages/torch_geometric/deprecation.py:13: UserWarning: 'data.DataLoader' is deprecated, use 'loader.DataLoader' instead warnings.warn(out) creating GNN model Traceback (most recent call last): File "run_image.py", line 340, in main(opt) File "run_image.py", line 164, in main model = GNN_image(opt, batch, opt['num_class'], device).to(device) File "/home/graph-neural-pde/src/GNN_image.py", line 22, in init range(self.n_ode_blocks)]).to(self.device) File "/home/anaconda3/envs/grand_2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 948, in getattr type(self).name, name)) AttributeError: 'GNN_image' object has no attribute 'n_ode_blocks'