❓ Questions

For example, when I fine-tune pretrained Vit model with LSQ on CIFAR-10 dataset, the output True model size is 41.20 MB, but on the ./outputs folder, the checkpoint.th is 686 MB, why is not in compliance with true model size?

❓ Questions

I tried to use DiffQ with PyTorch Lightning and found that the number of parameters doubled. Is this behavior expected? I am forced to reduce the batch size to avoid out-of-memory error. Minimal example:

import os

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, Dataset
import pytorch_lightning as pl
from sklearn.datasets import make_classification

from diffq import DiffQuantizer


class DataSet(Dataset):
    def __init__(self):
        self.X, self.y = make_classification(
            n_samples=100, 
            n_features=512, 
            shuffle=True, 
            random_state=0,
        )
  
    def __len__(self):
        return len(self.X)
    
    def __getitem__(self, i):
        x = torch.tensor(self.X[i]).float()
        y = torch.tensor(self.y[i]).long()
        return {'x': x, 'y': y}
    
    
class Model(pl.LightningModule):
    def __init__(self, q=True):
        super().__init__()
        self.q = q
        self.l1 = nn.Linear(512, 2**12)
        self.l2 = nn.Linear(2**12, 2)
        self.criterion = nn.NLLLoss()
        
    def forward(self, x): 
        y = self.l1(x)
        y = self.l2(y)
        return y
    
    def training_step(self, batch, batch_idx):
        y = batch['y']
        y_ = self(batch['x'])
        loss = self.criterion(y_,y)
        if self.q:
            loss += 1e-3 * self.quantizer.model_size()
        return loss
    
    def configure_optimizers(self):
        opt = torch.optim.Adam(self.parameters(), lr=1e-5)
        if self.q:
            self.quantizer = DiffQuantizer(self)
            self.quantizer.setup_optimizer(opt) 
        return {'optimizer': opt}
    
    
loader = DataLoader(
    DataSet(),
    batch_size=32,
    num_workers=os.cpu_count()
)

model = Model(q=bool(0))
trainer = pl.Trainer(max_epochs=2)
trainer.fit(model, loader)

model = Model(q=bool(1))
trainer = pl.Trainer(max_epochs=2)
trainer.fit(model, loader)

❓ Questions

Hi, I want to apply DiffQ to my source separation model with PyTorch Lightning framework, and I added the quantizer following the Usage on ReadMe.

In my callback function during training, it works fine when evaluating the unquantized model with SDR. But when I use the quantized model to run separation on MUSDB test set or other pop songs, the separation result is NaN or contains lots of 0s.

Do you have any comments or suggestions on this? Hope to get your reply! Sincerely,

❓ Questions

(Please ask your question here.) Hi after studying code, I found that the UniformQuantizer quantize the layer weights using forward pre hook But what confused me is that, I have not found where is the quantize operation on activation. seems that the hook do not operate the input Thanks

🐛 Bug Report

run cd examples/cifar/ and run flowing readme, but got the flowing error:

envs/diffq/lib/python3.7/site-packages/hydra/_internal/defaults_list.py:389: UserWarning: In config.yaml: Invalid over riding of hydra/job_logging: Default list overrides requires 'override' keyword. See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/defaults_list_override for more information.

change examples/cifar/conf/config.yaml from   1 defaults:   2   - hydra/job_logging: colorlog   3   - hydra/hydra_logging: colorlog

into

  1 defaults:   2   - override hydra/job_logging: colorlog   3   - override hydra/hydra_logging: colorlog can fix the issue

To Reproduce

(Write your steps here:)

pip install . make examples cd examples/cifar pip install -r requirements ./train.py db.name=cifar100 model=mobilenet quant.bits=3 quant.qat=True

Expected behavior

(Write what you thought would happen.)

Actual Behavior

(Write what happened. Add screenshots, if applicable.)

Your Environment

• Python and PyTorch version:
• Operating system and version (desktop or mobile):
• Hardware (gpu or cpu, amount of RAM etc.):