To visualize your network architecture more clearly, I intend to save the network architecture including model and bifpn into a tensorboard.

I add the following codes into your classification/main.py:

writer = SummaryWriter('./tensorboard/'+args_path) images, labels = next(iter(train_loader)) images = images.cuda() writer.add_graph(model, images) writer.add_graph(bifpn, images)

However, it returns many erros.

Therefore, I want to find some help from you to visualize your network architecture using a tensorboard.

When Training with FRSKD,

In train, the first feat has been deleted in feat_lst.

https://github.com/MingiJi/FRSKD/blob/bed26010383cbf4b8ae3a5d38c414d052c68a6ed/classification/utils.py#L60-L61

But, in model bifpn forward, it still deletes the first feat.

https://github.com/MingiJi/FRSKD/blob/bed26010383cbf4b8ae3a5d38c414d052c68a6ed/classification/models/bifpn.py#L27-L28

My guess, based on network_channels, is that only one delete operation is required.

https://github.com/MingiJi/FRSKD/blob/bed26010383cbf4b8ae3a5d38c414d052c68a6ed/classification/main.py#L75-L76

So, the following line should also be deleted

https://github.com/MingiJi/FRSKD/blob/bed26010383cbf4b8ae3a5d38c414d052c68a6ed/classification/models/bifpn.py#L28

'Weselect the attention maps at the 50-th epoch to observe thedistillation behaviors in the learning process' . As you said in your paper, but i can't visualization attention map like you did.Can you tell me how to code it? Thanks.

Hi,

Thanks for your contributions! When I tried to implement your code on Resnet50 and Resnet101, the loss value is nan. I just modified your code like this: def cifarresnet50(pretrained=False, **kwargs): return CIFAR_ResNet(Bottleneck, [3, 4, 6, 3], **kwargs) Then when I tried to train this network, the loss value is always nan.

(GPU: RTX 3090)

How to fix this error? Thanks!

I'm a little confused about whether it's a data problem or a code problem Here is the log information： ,epoch,train_loss,mIOU,time 0,1,0.7561731152236462,0.03367172123101207,172.40587258338928 1,2,0.748881900539765,0.03683467247028115,174.69739818572998 2,3,0.7231125767127826,0.03413524813319142,176.4995617866516 3,4,0.7103231128018636,0.034718840754135,175.68166756629944 4,5,0.6948323705448554,0.03985984205367289,176.37116837501526 5,6,0.6963401340091457,0.034812881275243984,175.4205687046051 6,7,0.670259090176282,0.039593741565003365,177.59505224227905 7,8,0.657163177903455,0.04566971232266198,177.0455219745636 8,9,0.6466049935955268,0.04648490031142609,175.9458737373352 9,10,0.6441954008948344,0.040296921257545505,175.44736456871033 10,11,0.6518893415251603,0.040108131647718384,173.6164116859436 11,12,0.6342953191353724,0.04960548057842536,174.7338092327118 12,13,0.627325470559299,0.04964727835046585,176.74344968795776 13,14,0.6132798489326468,0.05281939904056167,175.50420832633972 14,15,0.6002895892239534,0.055861818521504104,176.07024669647217 15,16,0.6025141052042062,0.05466266379947758,177.30515456199646 16,17,0.6029521900300796,0.04238770927785353,175.8492751121521 17,18,0.5934671195080647,0.07225978056218663,173.75626182556152 18,19,0.5773926787078381,0.048255481604236414,174.74913358688354 19,20,0.582425432673727,0.059308266458931364,176.88284063339233 20,21,0.5682540397661237,0.05038267338739297,174.98791122436523 21,22,0.5746843272533554,0.05538608518532103,173.91722106933594 22,23,0.5666266437619925,0.08541678590371297,174.55052709579468 23,24,0.5560441032911723,0.06600933025918862,175.00977063179016 24,25,0.5517762127833871,0.06934330535863947,178.28503155708313 25,26,0.5430047279940202,0.07609816247099357,173.69276547431946 26,27,0.5501473788888409,0.08342366132038433,177.11464619636536 27,28,0.5378253686313446,0.09859108165162175,176.39698958396912 28,29,0.5331599443721083,0.0678868360658615,176.34525656700134 29,30,0.5324760234126678,0.10438792503626895,176.21589493751526 30,31,0.5238752497646672,0.1128183428160334,175.78917503356934 31,32,0.5251926730315273,0.12039694674205649,176.04383778572083 32,33,0.5220876589345818,0.09766672767178268,174.95863962173462 33,34,0.5175259400588962,0.10621496236219143,176.0151607990265 34,35,0.5067871576175094,0.12684976035995407,177.03462433815002 35,36,0.49546105708353794,0.11654276929771637,176.7458143234253 36,37,0.494680989605303,0.124827549423532,176.56689929962158 37,38,0.506546473918626,0.13644115255903136,176.26826405525208 38,39,0.49985619039776236,0.13591350946832806,175.53716468811035 39,40,0.4863511062083909,0.1471462457362383,172.9881386756897 40,41,0.47111362118560535,0.16850882157957298,178.14349603652954 41,42,0.4635956700389775,0.17315480519723986,174.8266270160675 42,43,0.47008128091692924,0.1753281237951893,177.32808256149292 43,44,0.46045722468541217,0.16734526537605668,181.26892566680908 44,45,0.47051489163333404,0.1779139305812485,176.50252866744995 45,46,0.4693494217040447,0.1737308142689212,173.95223832130432 46,47,0.4636622975007273,0.17580288567626196,174.41174721717834 47,48,0.47545362876441616,0.18437971651423307,175.14617395401 48,49,0.48231911250891596,0.17996943729784323,175.4971604347229 49,50,0.4861245102678927,0.1646887044495716,172.72737431526184 50,51,0.4998336468751614,0.1784439646198613,178.50492978096008 51,52,0.5165176399481984,0.17607720511222485,177.06681728363037 52,53,0.5195794134902266,0.17282210617745608,177.14761924743652 53,54,0.5199347996654419,0.17748172026042575,173.87622380256653 54,55,0.527843453706457,0.17620730130932943,174.56411957740784 55,56,0.5464535670784804,0.1823876403320441,175.65909218788147 56,57,0.5455715203514466,0.17574022854965982,175.3632197380066 57,58,0.5364228865275016,0.18073436845138688,173.09280729293823 58,59,0.536486465913745,0.17925631794081523,181.71167612075806 59,60,0.5356380167202308,0.17350742104516761,174.03977036476135