Hi. Posting this over here instead of fastai 👍 I'm seeing the same exact issue all of the sudden: RuntimeError: running_mean should contain 1024 elements not 2048. https://colab.research.google.com/drive/1kY_qH-GCG9IiUhQmH0Gk3-IU338bD0Ea?usp=sharing

Thanks! Charlie

Since the issue yesterday and today, my models just running resnet50 on an image set is taking 4 hours an epoch. It used to take about 4 minutes. I'm running GPU and higher RAM on colabs. I've verified GPU is running. Is something up?

Hi. I'm pulling in: from wwf.vision.timm import *

When I ran your example: timm.list_models('efficientnet')[:10]

I see various models but for some reason even though I see b4 when I try to run it I get: Pretrained model URL is invalid, using random initialization. <fastai.learner.Learner at 0x7fe259fdd9b0>

But I don't get that with any b3 models.

This is my learn: learn = timm_learner(dls, 'efficientnet_b4', loss_func=LabelSmoothingCrossEntropy(), metrics=[accuracy], opt_func=ranger)

I ran a notebook last night, woke up this morning and re-ran it and it is giving me this error: RuntimeError: running_mean should contain 4304 elements not 8608

Here is what happened with my frozen parameters.

Originally posted by @craine in https://github.com/walkwithfastai/walkwithfastai.github.io/issues/18#issuecomment-945710718

This is basically the code added (by @muellerzr)

def __getitem__(self:Interpretation, idxs):
    "Get the inputs, preds, targets, decoded outputs, and losses at `idx`"
    if not is_listy(idxs): idxs = [idxs]
    attrs = 'inputs,preds,targs,decoded,losses'
    res = L([getattr(self, attr)[idxs] for attr in attrs.split(',')])
    return res

@patch
def show_at(self:Interpretation, idx:int, **kwargs):
    inp, _, targ, dec, _ = self[idx]
    dec, targ = L(dec, targ).map(Self.unsqueeze(0))
    self.dl.show_results((inp, dec), targ, **kwargs)

I just showcased it following in a nb following the format of this repo. The methods are exported to the module basics.interp.py

This PR is adding enhancements to the lr_find of fastai, with a tutorial that goes along with it.

The lr_find enhancement adds a visual marker on the lr_find graph to indicate where lr_min and lr_steep are. These are not new calculations, just added them to the graph as well.

Hi Zach, This is the small PR I mentioned on Discord, it fixes 2 minor inconsistencies:

1. Description of the convolutional layer that you used uses 5x5 kernel and (more importantly) stride=1 resulting in 28x28 output layer of the first convolution. In the model you build below, the first layer has shape 14x14 due to stride=2. I added paragraph that provides a bit of clarification to those differences as well as link to more in-depth explanation of conv layers
2. In the first net you build, you use the nn.Conv2d layer from PyTorch, which has bias on by default. This results in (3x3 +1) x 8 = 80 parameters for the first layer instead of expected 72 (other layers are altered the same way). Later in the notebook you use ConvLayer from FastAI, which has bias=None, resulting in expected number of params. I added bias=False to the initial conv definition for consistency.

There were also 1 or 2 typos in the notebook.

Following is my code from fastai.vision.all import * from wwf.vision.timm import * learn = load_learner('models/predict.pkl')

• gets following error
• No module named 'timm.models.layers.se' Full error report

ModuleNotFoundError Traceback (most recent call last) in ----> 1 learn = load_learner('models/predict.pkl')

~/anaconda3/envs/fastjul/lib/python3.8/site-packages/fastai/learner.py in load_learner(fname, cpu, pickle_module) 382 "Load a Learner object in fname, optionally putting it on the cpu" 383 distrib_barrier() --> 384 res = torch.load(fname, map_location='cpu' if cpu else None, pickle_module=pickle_module) 385 if hasattr(res, 'to_fp32'): res = res.to_fp32() 386 if cpu: res.dls.cpu()

~/anaconda3/envs/fastjul/lib/python3.8/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args) 605 opened_file.seek(orig_position) 606 return torch.jit.load(opened_file) --> 607 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) 608 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) 609

~/anaconda3/envs/fastjul/lib/python3.8/site-packages/torch/serialization.py in _load(zip_file, map_location, pickle_module, pickle_file, **pickle_load_args) 880 unpickler = UnpicklerWrapper(data_file, **pickle_load_args) 881 unpickler.persistent_load = persistent_load --> 882 result = unpickler.load() 883 884 torch._utils._validate_loaded_sparse_tensors()

~/anaconda3/envs/fastjul/lib/python3.8/site-packages/torch/serialization.py in find_class(self, mod_name, name) 873 def find_class(self, mod_name, name): 874 mod_name = load_module_mapping.get(mod_name, mod_name) --> 875 return super().find_class(mod_name, name) 876 877 # Load the data (which may in turn use persistent_load to load tensors)

ModuleNotFoundError: No module named 'timm.models.layers.se'

I am trying to deploy fastai trained model (build using wwf(walk with fastai), and time), to sagemaker endpoint. I have also included in required libraries as requirements.txt. But the model is not being able to loaded by load_learner:

This is my model-fn

![Screenshot 2021-05-26 160613](https://user-images.githubusercontent.com/62832721/119653690-7ba81580-bddc-11eb-84be-fa6757e9825b.png)

# loads the model into memory from disk and returns it
def model_fn(model_dir):
  logger.info('model_fn')
  path_model = Path(model_dir)
  logger.debug(f'Loading model from path: {str(path_model/EXPORT_MODEL_NAME)}')
  print(path_model/EXPORT_MODEL_NAME)
  defaults.device = torch.device('cpu')
  print("Trying to Load Model::::")
  learn = load_learner(path_model/EXPORT_MODEL_NAME, cpu=True)
  print('MODEL-LOADED')
  logger.info('model loaded successfully')
  return learn

So, in the screenshot it can be seen that "MODEL-LOADED" is not printed. Comparing, for a simple fastai-trained model, as a demo, that is working.

So kindly help me with this.

Thanks alot.

transformers+fastai classification tutorial includes:

• Brief description of GLUE tasks
• Batched tokenize transform
• How to make show_batch and show_results work
• Customized Learner subclass

Hi,

I am trying to run an image regression task.

dls = ImageDataLoaders.from_df(train, path = Path('./train'),fn_col='fn',label_col='wind_speed',y_block=RegressionBlock,seed=2020,item_tfms=Resize(224),batch_tfms=aug_transforms(size=224),bs=15) dls.show_batch()

Above code works fine.

But when I run the below code, training finishes smoothly but start of validation causes the error -

grid_sampler(): expected input and grid to have same dtype, but input has float and grid has c10::Half

May be related to issues with pytorch

`archs = ['efficientnet']

prediction_array = np.zeros(shape=(test.shape[0],1))

for arch in archs: print('Model ',arch) cbs=SaveModelCallback(every_epoch=False,fname='/content/drive/My Drive/wind-speed/models/effnet') learn = timm_learner(dls, 'efficientnet_b3',loss_func=MSELossFlat(),metrics=[rmse],cbs=cbs) #learn = cnn_learner(dls,arch,loss_func=MSELossFlat(),metrics=[rmse],cbs=cbs).to_fp16() learn.fine_tune(5) tdl = learn.dls.test_dl(test) preds = learn.get_preds(dl=tdl) prediction_array += preds[0].numpy() print('Prediction completed !!!')`

Model efficientnet

0.00% [0/1 00:00<00:00] epoch train_loss valid_loss _rmse time

0.00% [0/937 00:00<00:00]

RuntimeError Traceback (most recent call last) in () 10 learn = timm_learner(dls, 'efficientnet_b3',loss_func=MSELossFlat(),metrics=[rmse],cbs=cbs) 11 #learn = cnn_learner(dls,arch,loss_func=MSELossFlat(),metrics=[rmse],cbs=cbs).to_fp16() ---> 12 learn.fine_tune(5) 13 tdl = learn.dls.test_dl(test) 14 preds = learn.get_preds(dl=tdl)

28 frames /usr/local/lib/python3.6/dist-packages/fastai/callback/schedule.py in fine_tune(self, epochs, base_lr, freeze_epochs, lr_mult, pct_start, div, **kwargs) 155 "Fine tune with freeze for freeze_epochs then with unfreeze from epochs using discriminative LR" 156 self.freeze() --> 157 self.fit_one_cycle(freeze_epochs, slice(base_lr), pct_start=0.99, **kwargs) 158 base_lr /= 2 159 self.unfreeze()

/usr/local/lib/python3.6/dist-packages/fastai/callback/schedule.py in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt) 110 scheds = {'lr': combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final), 111 'mom': combined_cos(pct_start, *(self.moms if moms is None else moms))} --> 112 self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd) 113 114 # Cell

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt) 209 self.opt.set_hypers(lr=self.lr if lr is None else lr) 210 self.n_epoch = n_epoch --> 211 self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup) 212 213 def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in with_events(self, f, event_type, ex, final) 158 159 def with_events(self, f, event_type, ex, final=noop): --> 160 try: self(f'before{event_type}'); f() 161 except ex: self(f'after_cancel{event_type}') 162 self(f'after_{event_type}'); final()

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in _do_fit(self) 200 for epoch in range(self.n_epoch): 201 self.epoch=epoch --> 202 self._with_events(self._do_epoch, 'epoch', CancelEpochException) 203 204 def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False):

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in with_events(self, f, event_type, ex, final) 158 159 def with_events(self, f, event_type, ex, final=noop): --> 160 try: self(f'before{event_type}'); f() 161 except ex: self(f'after_cancel{event_type}') 162 self(f'after_{event_type}'); final()

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in _do_epoch(self) 195 def _do_epoch(self): 196 self._do_epoch_train() --> 197 self._do_epoch_validate() 198 199 def _do_fit(self):

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in _do_epoch_validate(self, ds_idx, dl) 191 if dl is None: dl = self.dls[ds_idx] 192 self.dl = dl --> 193 with torch.no_grad(): self._with_events(self.all_batches, 'validate', CancelValidException) 194 195 def _do_epoch(self):

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in with_events(self, f, event_type, ex, final) 158 159 def with_events(self, f, event_type, ex, final=noop): --> 160 try: self(f'before{event_type}'); f() 161 except ex: self(f'after_cancel{event_type}') 162 self(f'after_{event_type}'); final()

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in all_batches(self) 164 def all_batches(self): 165 self.n_iter = len(self.dl) --> 166 for o in enumerate(self.dl): self.one_batch(*o) 167 168 def _do_one_batch(self):

/usr/local/lib/python3.6/dist-packages/fastai/data/load.py in iter(self) 101 for b in _loadersself.fake_l.num_workers==0: 102 if self.device is not None: b = to_device(b, self.device) --> 103 yield self.after_batch(b) 104 self.after_iter() 105 if hasattr(self, 'it'): del(self.it)

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in call(self, o) 196 self.fs.append(t) 197 --> 198 def call(self, o): return compose_tfms(o, tfms=self.fs, split_idx=self.split_idx) 199 def repr(self): return f"Pipeline: {' -> '.join([f.name for f in self.fs if f.name != 'noop'])}" 200 def getitem(self,i): return self.fs[i]

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in compose_tfms(x, tfms, is_enc, reverse, **kwargs) 148 for f in tfms: 149 if not is_enc: f = f.decode --> 150 x = f(x, **kwargs) 151 return x 152

/usr/local/lib/python3.6/dist-packages/fastai/vision/augment.py in call(self, b, split_idx, **kwargs) 33 def call(self, b, split_idx=None, **kwargs): 34 self.before_call(b, split_idx=split_idx) ---> 35 return super().call(b, split_idx=split_idx, **kwargs) if self.do else b 36 37 # Cell

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in call(self, x, **kwargs) 71 @property 72 def name(self): return getattr(self, '_name', _get_name(self)) ---> 73 def call(self, x, **kwargs): return self._call('encodes', x, **kwargs) 74 def decode (self, x, **kwargs): return self._call('decodes', x, **kwargs) 75 def repr(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs) 81 def _call(self, fn, x, split_idx=None, **kwargs): 82 if split_idx!=self.split_idx and self.split_idx is not None: return x ---> 83 return self._do_call(getattr(self, fn), x, **kwargs) 84 85 def _do_call(self, f, x, **kwargs):

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in do_call(self, f, x, **kwargs) 88 ret = f.returns(x) if hasattr(f,'returns') else None 89 return retain_type(f(x, **kwargs), x, ret) ---> 90 res = tuple(self.do_call(f, x, **kwargs) for x in x) 91 return retain_type(res, x) 92

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in (.0) 88 ret = f.returns(x) if hasattr(f,'returns') else None 89 return retain_type(f(x, **kwargs), x, ret) ---> 90 res = tuple(self.do_call(f, x, **kwargs) for x_ in x) 91 return retain_type(res, x) 92

/usr/local/lib/python3.6/dist-packages/fastcore/transform.py in do_call(self, f, x, **kwargs) 87 if f is None: return x 88 ret = f.returns(x) if hasattr(f,'returns') else None ---> 89 return retain_type(f(x, **kwargs), x, ret) 90 res = tuple(self.do_call(f, x, **kwargs) for x in x) 91 return retain_type(res, x)

/usr/local/lib/python3.6/dist-packages/fastcore/dispatch.py in call(self, *args, **kwargs) 116 elif self.inst is not None: f = MethodType(f, self.inst) 117 elif self.owner is not None: f = MethodType(f, self.owner) --> 118 return f(*args, **kwargs) 119 120 def get(self, inst, owner):

/usr/local/lib/python3.6/dist-packages/fastai/vision/augment.py in encodes(self, x) 397 return x.affine_coord(self.mat, coord_func, sz=self.size, mode=mode, pad_mode=self.pad_mode, align_corners=self.align_corners) 398 --> 399 def encodes(self, x:TensorImage): return self._encode(x, self.mode) 400 def encodes(self, x:TensorMask): return self._encode(x, self.mode_mask) 401 def encodes(self, x:(TensorPoint, TensorBBox)): return self._encode(x, self.mode, reverse=True)

/usr/local/lib/python3.6/dist-packages/fastai/vision/augment.py in _encode(self, x, mode, reverse) 395 def _encode(self, x, mode, reverse=False): 396 coord_func = None if len(self.coord_fs)==0 or self.split_idx else partial(compose_tfms, tfms=self.coord_fs, reverse=reverse) --> 397 return x.affine_coord(self.mat, coord_func, sz=self.size, mode=mode, pad_mode=self.pad_mode, align_corners=self.align_corners) 398 399 def encodes(self, x:TensorImage): return self._encode(x, self.mode)

/usr/local/lib/python3.6/dist-packages/fastai/vision/augment.py in affine_coord(x, mat, coord_tfm, sz, mode, pad_mode, align_corners) 319 coords = affine_grid(mat, x.shape[:2] + size, align_corners=align_corners) 320 if coord_tfm is not None: coords = coord_tfm(coords) --> 321 return TensorImage(_grid_sample(x, coords, mode=mode, padding_mode=pad_mode, align_corners=align_corners)) 322 323 @patch

/usr/local/lib/python3.6/dist-packages/fastai/vision/augment.py in _grid_sample(x, coords, mode, padding_mode, align_corners) 304 else: 305 x = F.interpolate(x, scale_factor=1/d, mode='area') --> 306 return F.grid_sample(x, coords, mode=mode, padding_mode=padding_mode, align_corners=align_corners) 307 308 # Cell

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in grid_sample(input, grid, mode, padding_mode, align_corners) 3361 return handle_torch_function( 3362 grid_sample, tens_ops, input, grid, mode=mode, padding_mode=padding_mode, -> 3363 align_corners=align_corners) 3364 if mode != 'bilinear' and mode != 'nearest': 3365 raise ValueError("nn.functional.grid_sample(): expected mode to be "

/usr/local/lib/python3.6/dist-packages/torch/overrides.py in handle_torch_function(public_api, relevant_args, *args, **kwargs) 1061 # Use public_api instead of implementation so torch_function 1062 # implementations can do equality/identity comparisons. -> 1063 result = overloaded_arg.torch_function(public_api, types, args, kwargs) 1064 1065 if result is not NotImplemented:

/usr/local/lib/python3.6/dist-packages/fastai/torch_core.py in torch_function(self, func, types, args, kwargs) 323 convert=False 324 if _torch_handled(args, self._opt, func): convert,types = type(self),(torch.Tensor,) --> 325 res = super().torch_function(func, types, args=args, kwargs=kwargs) 326 if convert: res = convert(res) 327 if isinstance(res, TensorBase): res.set_meta(self, as_copy=True)

/usr/local/lib/python3.6/dist-packages/torch/tensor.py in torch_function(cls, func, types, args, kwargs) 993 994 with _C.DisableTorchFunction(): --> 995 ret = func(*args, **kwargs) 996 return _convert(ret, cls) 997

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in grid_sample(input, grid, mode, padding_mode, align_corners) 3389 align_corners = False 3390 -> 3391 return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum, align_corners) 3392 3393

RuntimeError: grid_sampler(): expected input and grid to have same dtype, but input has float and grid has c10::Half

Bumps nokogiri from 1.12.5 to 1.13.9.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps tzinfo from 1.2.9 to 1.2.10.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

https://walkwithfastai.com/Object_Detection

Using the current version of fastai, attempting to show a batch produces this error:

RuntimeError: Error when trying to collate the data into batches with fa_collate, at least two tensors in the batch are not the same size.

Mismatch found on axis 1 of the batch and is of type `TensorBBox`:
	Item at index 0 has shape: torch.Size([6, 4])
	Item at index 1 has shape: torch.Size([1, 4])

Please include a transform in `after_item` that ensures all data of type TensorBBox is the same size

If I install the versions listed on the page (fastai==2.1.10 fastcore==1.3.13 wwf==0.0.8), I get an initial import error: ModuleNotFoundError: No module named 'torchvision.models.utils' ... for which there is a workaround, which then leaves me with this error for pascal.summary:

TypeError: no implementation found for 'torch.Tensor.__getitem__' on types that implement __torch_function__: [<class 'fastai.torch_core.TensorMultiCategory'>, <class 'fastai.vision.core.TensorBBox'>]

There is a typo in the 'Single Label Classification with Computer Vision (Beginner)'documentation. Instead of 'using only', 'usin gonly' is there just below the training code blocks.

@muellerzr Adding this issue based on fastai-dev discussion . I would like to use this issue to learn about callbacks. I liked your idea about having a MemoryMangerCallback callback for this purpose.

https://twitter.com/jeanmarcalkazzi/status/1465418318068781062?s=20 https://github.com/rentruewang/koila

self-assign

currently using unet_config . An obsolete method for learner params

#Problem: cfg = unet_config(blur=True, norm_type=NormType.Weight, self_attention=True, y_range=y_range)

def create_gen_learner(): return unet_learner(dls_gen, bbone, loss_func=loss_gen, config=cfg)

#Fix: remove cfg lne & Feed the params in directly to the learner def create_gen_learner(): return unet_learner(dls_gen, bbone, loss_func=loss_gen, blur=True, norm_type=NormType.Weight, self_attention=True, y_range=y_range)