Hello, I am trying to reproduce the results for the other datasets as well, for Cora dataset for example, what is the train/test split you have used (this dataset is for node-classification), I am not aware of formal test/train split, did you take some percentage as train and the rest as test? any clarification about this will help a lot Thansk!

Hi I am trying to reproduce your results on reddit dataset using the 4 BITS configuration. I run the following command : python reddit_binary/main.py --int4 --gc_per --lr 0.001 --DQ --low 0.1 --change 0.1 --wd 4e-5 --epochs 200 --outdir ${RUN_DIR} --path ${DATA_DIR} | tee int4_dq.txt However the results I get are 86.4, The paper says that for this config it should give 81.3 (table 3) am I missing something ? Thanks !

Thank you for releasing the code! Found a bug in how transforms are composed for Degree-Quant when I customized the REDDIT example code to my own dataset:

https://github.com/camlsys/degree-quant/blob/5f056017bd0f4b1018842fb1a083ccf6c666e0b4/reddit_binary/dataset.py#L71

Had to change to:

if dataset.transform is None:
    dataset.transform = dq_transform
else:
    dataset.transform = T.Compose([dataset.transform, dq_transform])

Otherwise, the transform becomes a list [None, ProbabilisticHighDegreeMask].

Hello, Your paper seems to give the int8 model running time on the GPU in table 4, and the time is reduced compared to float32. Does this code describe how to get this time?

I have tested the "val_loss time" in this code, but the FP32 model seems to be faster than the INT8 model.

torch.cuda.synchronize()
s = time.time()
val_loss = eval_loss(model, val_loader)
torch.cuda.synchronize()
e = time.time()
print(e-s)

Could you tell me how I should measure the time if my method is wrong?

Thank you so much for posting the code!

I am just facing a lot of issues trying to extend your code for GCNs. I created a file for gcn.py similar to gin.py and passed the in and out channels for GCN accordingly and adjusted the dataset. I am now getting an error regarding the keyword "weights_low" as follows:

Traceback (most recent call last): File "degree/reddit_binary/main.py", line 109, in Namespace(DQ=True, batch_size=128, change=0.1, epochs=200, fp32=False, gc_abs=False, gc_mom=False, gc_per=True, hidden=64, int4=False, int8=True, low=0.0, lr=0.005, lr_decay_factor=0.5, lr_decay_step_size=50, noise=1.0, num_layers=5, outdir='./run', path='./data', sample_prop=None, ste_abs=False, ste_mom=False, ste_per=False, wd=0.0002) Generating ProbabilisticHighDegreeMask: {'prob_mask_low': 0.0, 'prob_mask_change': 0.1} model = GCN( File "/content/degree/reddit_binary/gcn.py", line 134, in init self.conv1 = gcn_layer( File "/content/degree/dq/multi_quant.py", line 247, in init self.reset_parameters() File "/content/degree/dq/multi_quant.py", line 259, in reset_parameters self.layer_quantizers[key] = self.layer_quant_fnskey KeyError: 'weights_low'

Can you please help with this?

Thanks a lot for posting the code! I'm quite new to GNN, and I am facing errors in training the GIN model with degree quant support with the OGB-MOLHIV binary classification dataset. However, I face an issue in the below line: https://github.com/camlsys/degree-quant/blob/257d6bcf25141f522d0dbfdd2ae9fdabb679f7e0/dq/quantization.py#L57

The error is:

RuntimeError: result type Float can't be cast to the desired output type Long

The inv_scale and zero_point values I obtained are 15.9375 and -128.0, respectively.

I have only changed the dataset inside the get_dataset() function. Do I need to consider anything else? I fixed the dataset transform bug as reported by issue #1

Please help me with how to resolve this.

Thanks!!