In distill_basic_teacher.ipynb, for both training_dataset and test_dataset, the transform.Normalize has been set wrongly. The mentioned transform converts the input image to the shape (2,28,28).

Hello,

I followed the instruction and trained the teacher model successfully. However, when I tried to do the same with the student model it showed me this error:

nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# plt.rcParams['figure.figsize'] = [10, 5]
weight_decay_scatter = ([math.log10(h['weight_decay']) if h['weight_decay'] > 0 else -6 for h in hparams_list])
dropout_scatter = [int(h['dropout_input'] == 0.2) for h in hparams_list]
colors = []
print('hparams_list has a length of ', len(hparams_list))
for i in range(len(hparams_list)):
    cur_hparam_tuple = utils.hparamDictToTuple(hparams_list[i])
    colors.append(results_no_distill[cur_hparam_tuple]['val_acc'][-1])

marker_size = 100
fig, ax = plt.subplots()
plt.scatter(weight_decay_scatter, dropout_scatter, marker_size, c=colors, edgecolors='black')
plt.colorbar()
print('weight_decay_scatter has a length of ', len(weight_decay_scatter))
print('colors has a length of ', len(colors))
print('dropout_scatter has a length of ', len(dropout_scatter))
for i in range(len(weight_decay_scatter)):
    ax.annotate(str('%0.4f' % (colors[i], )), (weight_decay_scatter[i], dropout_scatter[i]))
plt.show()

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-1-945da500c1fb> in <module>
      6 for i in range(len(hparams_list)):
      7     cur_hparam_tuple = utils.hparamDictToTuple(hparams_list[i])
----> 8     colors.append(results_no_distill[cur_hparam_tuple]['val_acc'][-1])
      9 
     10 marker_size = 100

IndexError: list index out of range
IndexError: list index out of range

I am unsure what caused this. I tried to debug by first printing out the length of each list/tuple that could cause this but the result would not print the length out.

Hi, I am facing issues while training the basic_teacher model.

Attaching the logs here-

Training with hparamsdropout_hidden=0.0, dropout_input=0.0, lr=0.01, lr_decay=0.95, momentum=0.9, weight_decay=1e-05 /usr/local/lib/python3.6/dist-packages/torch/optim/lr_scheduler.py:123: UserWarning: Detected call oflr_scheduler.step()beforeoptimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order:optimizer.step()beforelr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

RuntimeError Traceback (most recent call last) in () 30 train_val_loader, None, 31 print_every=print_every, ---> 32 fast_device=fast_device) 33 save_path = checkpoints_path + utils.hparamToString(hparam) + '_final.tar' 34 torch.save({'results' : results[hparam_tuple],

4 frames /usr/local/lib/python3.6/dist-packages/torch/_utils.py in reraise(self) 393 # (https://bugs.python.org/issue2651), so we work around it. 394 msg = KeyErrorMessage(msg) --> 395 raise self.exc_type(msg)

RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/mnist.py", line 97, in getitem img = self.transform(img) File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 61, in call img = t(img) File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 212, in call return F.normalize(tensor, self.mean, self.std, self.inplace) File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/functional.py", line 298, in normalize tensor.sub(mean).div(std) RuntimeError: output with shape [1, 28, 28] doesn't match the broadcast shape [2, 28, 28]`

Hi @shriramsb. Unfortunately, I can't reach the accuracy in the teacher model based on your code snippet. would you please take a look at it? Maybe the architecture is not appropriate.