Hi,

I try to run your code through

config=configs/molpcba/gnn-transformer/no-virtual/JK=cat/pooling=cls+gin+norm_input.yml


echo "python main.py --configs $config --num_workers 8 --devices $CUDA_VISIBLE_DEVICES"

/home/yiming/anaconda3/envs/graphtrans/bin/python main.py \
              --configs $config \
              --num_workers 8 \
              --data_root data/ogb \
              --devices $CUDA_VISIBLE_DEVICES

However, it shows the below error:

Traceback (most recent call last):
  File "main.py", line 280, in <module>
    main()
  File "main.py", line 155, in main
    train_loader_, train_loader_eval_, valid_loader_, test_loader_ = create_loader(dataset_, dataset_eval_)
  File "main.py", line 143, in create_loader
    test_data = compute_adjacency_list_cached(dataset[split_idx["test"]], key=f"{args.dataset}_test")
  File "/home/yiming/graphtrans/data/adj_list.py", line 45, in compute_adjacency_list_cached
    result = compute_adjacency_list(data)
  File "/home/yiming/graphtrans/data/adj_list.py", line 26, in compute_adjacency_list
    out.append(make_adj_list_wrapper(x))
  File "/home/yiming/graphtrans/data/adj_list.py", line 20, in make_adj_list_wrapper
    return make_adj_list(x["num_nodes"], x["edge_index"].T)
  File "/home/yiming/anaconda3/envs/graphtrans/lib/python3.8/site-packages/torch_geometric/data/data.py", line 371, in __getitem__
    return self._store[key]
  File "/home/yiming/anaconda3/envs/graphtrans/lib/python3.8/site-packages/torch_geometric/data/storage.py", line 70, in __getitem__
    return self._mapping[key]
KeyError: 'num_nodes'

May I ask how to fix this?

Thank you!

Yiming

Hi!

I'm trying to install the project dependencies from the conda environment file on macOS 12.3.1, and am seeing the below error. It's consistent with an open issue in conda, whereby including the package builds in the env file sometimes causes problems when the environment is created on a different architecture.

One solution seems to be exporting the environment requirements with the flag --no-builds. Is there any chance you could update the env file by running this, or is there a strong reason for including builds?

Thanks!

 conda env create -f requirement.yml                                                                     (base) 7.4s
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:
  - mkl_fft==1.3.0=py38h54f3939_0
  - python==3.8.8=hdb3f193_4
  - ld_impl_linux-64==2.33.1=h53a641e_7
  - libstdcxx-ng==9.1.0=hdf63c60_0
  - libpng==1.6.37=hbc83047_0
  - libgcc-ng==9.1.0=hdf63c60_0
  - certifi==2020.12.5=py38h06a4308_0
  - ca-certificates==2021.1.19=h06a4308_1
  - numpy==1.19.2=py38h54aff64_0
  - ninja==1.10.2=py38hff7bd54_0
  - libuv==1.40.0=h7b6447c_0
  - intel-openmp==2020.2=254
  - torchvision==0.8.2=py38_cu102
  - pip==21.0.1=py38h06a4308_0
  - openssl==1.1.1j=h27cfd23_0
  - mkl_random==1.1.1=py38h0573a6f_0
  - setuptools==52.0.0=py38h06a4308_0
  - tk==8.6.10=hbc83047_0
  - sqlite==3.33.0=h62c20be_0
  - mkl-service==2.3.0=py38he904b0f_0
  - numpy-base==1.19.2=py38hfa32c7d_0
  - pytorch==1.7.1=py3.8_cuda10.2.89_cudnn7.6.5_0
  - zlib==1.2.11=h7b6447c_3
  - lcms2==2.11=h396b838_0
  - cudatoolkit==10.2.89=hfd86e86_1
  - libedit==3.1.20191231=h14c3975_1
  - six==1.15.0=py38h06a4308_0
  - zstd==1.4.5=h9ceee32_0
  - libtiff==4.1.0=h2733197_1
  - xz==5.2.5=h7b6447c_0
  - readline==8.1=h27cfd23_0
  - freetype==2.10.4=h5ab3b9f_0
  - libffi==3.3=he6710b0_2
  - ncurses==6.2=he6710b0_1
  - jpeg==9b=h024ee3a_2
  - lz4-c==1.9.3=h2531618_0
  - mkl==2020.2=256
  - pillow==8.1.2=py38he98fc37_0

It looks like in the example commands used in the README, the double dash for the parameter was collapsed into a single hyphen: -- to –. This errors if copying and pasting the command.

Hello, Thanks for excellent work. But I have found some possible problems. In the paper, the authors mention that "In Table 5, we tested several common methods to for sequence classification. The mean operation averages the output embeddings of the transformer to a single graph embedding; the last operation takes the last embedding in the output sequence as the graph embedding."

Table 5: Model, Valid, Test GraphTrans-mean, 0.1398, 0.1509

However, I observed the GraphTrans code and found that the author's implementation of mean could be wrong. gnn_transformer.py, line 116-117: elif self.pooling == "mean": h_graph = transformer_out.sum(0) / src_padding_mask.sum(-1, keepdim=True) transformer_out.shape = (S, B, h_d), src_padding_mask.shape = (B, S) The padding nodes information in transformer_out, and the authors do not remove them (do not unpad_batch) but sum directly.

I modified the mean operation, then redid the experiment and found that the result is improved than the one reported by the authors.

Hi, Thanks for your excellent work. I found the implementation of padding a batch in your code is confusing. The code for batch is:

def pad_batch(h_node, batch, max_input_len, get_mask=False):
    num_batch = batch[-1] + 1
    num_nodes = []
    masks = []
    for i in range(num_batch):
        mask = batch.eq(i)
        masks.append(mask)
        num_node = mask.sum()
        num_nodes.append(num_node)

    # logger.info(max(num_nodes))
    max_num_nodes = min(max(num_nodes), max_input_len)
    padded_h_node = h_node.data.new(max_num_nodes, num_batch, h_node.size(-1)).fill_(0)
    src_padding_mask = h_node.data.new(num_batch, max_num_nodes).fill_(0).bool()

    for i, mask in enumerate(masks):
        num_node = num_nodes[i]
        if num_node > max_num_nodes:
            num_node = max_num_nodes
        padded_h_node[-num_node:, i] = h_node[mask][-num_node:]
        src_padding_mask[i, : max_num_nodes - num_node] = True  # [b, s]

    if get_mask:
        return padded_h_node, src_padding_mask, num_nodes, masks, max_num_nodes
    return padded_h_node, src_padding_mask

I think the line "src_padding_mask[i, : max_num_nodes - num_node] = True" for masking might should be:

src_padding_mask = h_node.data.new(num_batch, max_num_nodes).fill_(1).bool()
src_padding_mask[i, : max_num_nodes - num_node] = False

Because in the pooling part, the original code can cause the denominator of the this line as 0:

h_graph = transformer_out.sum(0) / src_padding_mask.sum(-1, keepdim=True)

Hi!

I'm trying to scale the GNN transformer model down so I can run it with less compute. I can change the GNN embedding dimension, number of layers, transformer feedforward dimension etc., but my network remains relatively large in number of parameters.

I think this is mostly from the last set of layers in the model, the Linears that I think correspond to the readout function. These layers' output dimension seems tied to the dataset; how can I scale these down?

Thanks! Andrei