Summary: D35519683 changed the default matching_heuristics from False to True, it may cause some issues:

• potential bug report: https://fb.workplace.com/groups/2240361332735959/posts/5264606250311437/?comment_id=5265167493588646
• bloated logging: https://github.com/facebookresearch/detectron2/issues/4364

This diff reverts the default value back to False, and introduce a way to specify this option via filename url.

Differential Revision: D41145857

After your recent release, I got this error from my code. I can only use my code by downgrading fvcore version to previous one.

Error Traceback:

2021-10-15T09:46:49.0347680Z ==================================== ERRORS ====================================
2021-10-15T09:46:49.0350615Z ________________________ ERROR collecting test session _________________________
2021-10-15T09:46:49.0352801Z /usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py:495: in _importconftest
2021-10-15T09:46:49.0358791Z     return self._conftestpath2mod[key]
2021-10-15T09:46:49.0360317Z E   KeyError: PosixPath('/app/cv_ner_detectron/tests/conftest.py')
2021-10-15T09:46:49.0360877Z 
2021-10-15T09:46:49.0362075Z During handling of the above exception, another exception occurred:
2021-10-15T09:46:49.0363832Z /usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py:501: in _importconftest
2021-10-15T09:46:49.0364711Z     mod = conftestpath.pyimport()
2021-10-15T09:46:49.0365834Z /usr/local/lib/python3.8/dist-packages/py/_path/local.py:704: in pyimport
2021-10-15T09:46:49.0368988Z     __import__(modname)
2021-10-15T09:46:49.0370340Z /usr/local/lib/python3.8/dist-packages/_pytest/assertion/rewrite.py:152: in exec_module
2021-10-15T09:46:49.0372267Z     exec(co, module.__dict__)
2021-10-15T09:46:49.0373441Z cv_ner_detectron/tests/conftest.py:11: in <module>
2021-10-15T09:46:49.0377209Z     from cv_ner_detectron.detectron.datamodels import CvNerExample
2021-10-15T09:46:49.0378686Z cv_ner_detectron/detectron/datamodels.py:7: in <module>
2021-10-15T09:46:49.0380548Z     from detectron2.structures import BoxMode
2021-10-15T09:46:49.0382497Z /usr/local/lib/python3.8/dist-packages/detectron2/structures/__init__.py:6: in <module>
2021-10-15T09:46:49.0385934Z     from .keypoints import Keypoints, heatmaps_to_keypoints
2021-10-15T09:46:49.0387443Z /usr/local/lib/python3.8/dist-packages/detectron2/structures/keypoints.py:6: in <module>
2021-10-15T09:46:49.0389906Z     from detectron2.layers import interpolate
2021-10-15T09:46:49.0391270Z /usr/local/lib/python3.8/dist-packages/detectron2/layers/__init__.py:10: in <module>
2021-10-15T09:46:49.0397234Z     from .blocks import CNNBlockBase, DepthwiseSeparableConv2d
2021-10-15T09:46:49.0402621Z /usr/local/lib/python3.8/dist-packages/detectron2/layers/blocks.py:4: in <module>
2021-10-15T09:46:49.0406940Z     import fvcore.nn.weight_init as weight_init
2021-10-15T09:46:49.0408469Z /usr/local/lib/python3.8/dist-packages/fvcore/nn/__init__.py:2: in <module>
2021-10-15T09:46:49.0409904Z     from .activation_count import ActivationCountAnalysis, activation_count
2021-10-15T09:46:49.0411437Z /usr/local/lib/python3.8/dist-packages/fvcore/nn/activation_count.py:10: in <module>
2021-10-15T09:46:49.0414217Z     from .jit_analysis import JitModelAnalysis
2021-10-15T09:46:49.0415640Z /usr/local/lib/python3.8/dist-packages/fvcore/nn/jit_analysis.py:15: in <module>
2021-10-15T09:46:49.0417905Z     from fvcore.common.checkpoint import _named_modules_with_dup
2021-10-15T09:46:49.0419295Z /usr/local/lib/python3.8/dist-packages/fvcore/common/checkpoint.py:23: in <module>
2021-10-15T09:46:49.0422403Z     from torch.quantization import ObserverBase, FakeQuantizeBase
2021-10-15T09:46:49.0424633Z E   ImportError: cannot import name 'FakeQuantizeBase' from 'torch.quantization' (/usr/local/lib/python3.8/dist-packages/torch/quantization/__init__.py)
2021-10-15T09:46:49.0425328Z 

The CropTransform currently leaves boxes hanging over the edge of the images. I think in most object detection pipelines I would want the boxes cropped just like the image.

import numpy as np

from fvcore.transforms import CropTransform

transform = CropTransform(5, 5, 5, 5)
box = np.array([[0, 0, 10, 10]])
transform.apply_box(box)
# get: array([[-5, -5,  5,  5]])
# wanted: array([[0, 0, 5, 5]])

🚀 Feature

Add option to keep only the most recent n checkpoints in PeriodicCheckpointer.

Motivation

Checkpoints are quite large and often only the most recent one is of interest.

Hi,

Thanks for releasing this helpful codebase. After reading the part of precise-bn, a question comes to me: Do I need to enlarge the batch size when I do precise bn? I noticed that the performance of bn becomes bad when the batch size is small. People say that the bad performance is due to the noisy estimation of running mean/var of the bn layers. When batch size is small, there will be more noise in each batch's mean/var which brings bad estimation for the running mean/var. So in order to cope with the problem of small batch, do I need to enlarge the batch size when I use precise bn, or is there other explanations of the bn performance drop?

Hey, I am using torch version 1.7.1 CPU and detectron2 along with torch 1.7 but when I used it through my own package, I started getting this error. Please tell me how should I fix this?

ImportError                               Traceback (most recent call last)
<ipython-input-4-d3af5af0e43a> in <module>()
----> 1 from INPR import inpr
      2 import matplotlib.pyplot as plt
      3 get_ipython().magic('matplotlib inline')

4 frames
/usr/local/lib/python3.7/dist-packages/INPR/inpr.py in <module>()
      4 from PIL import Image
      5 import numpy as np
----> 6 from .utils import Load_model
      7 from .get_num_plate import get_number_plate
      8 from .get_details import fetch

/usr/local/lib/python3.7/dist-packages/INPR/utils.py in <module>()
----> 1 from detectron2.engine import DefaultPredictor
      2 from detectron2.data import MetadataCatalog
      3 from detectron2.config import get_cfg
      4 from detectron2.utils.visualizer import ColorMode, Visualizer
      5 from detectron2 import model_zoo

/usr/local/lib/python3.7/dist-packages/detectron2/engine/__init__.py in <module>()
      9 # prefer to let hooks and defaults live in separate namespaces (therefore not in __all__)
     10 # but still make them available here
---> 11 from .hooks import *
     12 from .defaults import *

/usr/local/lib/python3.7/dist-packages/detectron2/engine/hooks.py in <module>()
     11 from collections import Counter
     12 import torch
---> 13 from fvcore.common.checkpoint import PeriodicCheckpointer as _PeriodicCheckpointer
     14 from fvcore.common.param_scheduler import ParamScheduler
     15 from fvcore.common.timer import Timer

/usr/local/lib/python3.7/dist-packages/fvcore/common/checkpoint.py in <module>()
     21 else:
     22     from torch import quantization
---> 23     from torch.quantization import ObserverBase, FakeQuantizeBase
     24 
     25 __all__ = ["Checkpointer", "PeriodicCheckpointer"]

ImportError: cannot import name 'FakeQuantizeBase' from 'torch.quantization' (/usr/local/lib/python3.7/dist-packages/torch/quantization/__init__.py)

Hi, Thank you for this super-cool code. I am using this code to calculate the flops of my model. However, I encountered a very strange problem as below. It seems that the overall flops of the whole model is less than, even some modules. Also, it seems that some conv layers don't calculate the flops. Why does this situation happen? I am wondering if there is any solution to help with this so strange problem. Thank you very much.

D:\anaconda3\envs\PyTorch_py3\lib\site-packages\fvcore\nn\jit_handles.py:139: RuntimeWarning: overflow encountered in long_scalars flop = batch_size * out_size * Cout_dim * Cin_dim * kernel_size D:\anaconda3\envs\PyTorch_py3\lib\collections_init_.py:802: RuntimeWarning: overflow encountered in long_scalars self[elem] += count | module | #parameters or shape | #flops | |:-----------------------------|:-----------------------|:----------| | model | 6.789M | 1.09G | | conv1.conv2d | 0.379M | 1.904G | | conv1.conv2d.weight | (256, 1478, 1, 1) | | | conv1.conv2d.bias | (256,) | | | conv2.conv2d | 0.59M | 1.074G | | conv2.conv2d.weight | (256, 256, 3, 3) | | | conv2.conv2d.bias | (256,) | | | conv3.conv2d | 0.59M | 0 | | conv3.conv2d.weight | (256, 256, 3, 3) | | | conv3.conv2d.bias | (256,) | | | res_module | 3.59M | 3.195M | | res_module.0 | 1.197M | 1.065M | | res_module.0.conv1.conv2d | 0.59M | 0 | | res_module.0.conv2.conv2d | 0.59M | 0 | | res_module.0.se_layer | 16.672K | 1.065M | | res_module.1 | 1.197M | 1.065M | | res_module.1.conv1.conv2d | 0.59M | 0 | | res_module.1.conv2.conv2d | 0.59M | 0 | | res_module.1.se_layer | 16.672K | 1.065M | | res_module.2 | 1.197M | 1.065M | | res_module.2.conv1.conv2d | 0.59M | 0 | | res_module.2.conv2.conv2d | 0.59M | 0 | | res_module.2.se_layer | 16.672K | 1.065M | | deconv1.conv2d | 1.049M | 0 | | deconv1.conv2d.weight | (256, 256, 4, 4) | | | deconv1.conv2d.bias | (256,) | | | deconv2.conv2d | 0.59M | 1.074G | | deconv2.conv2d.weight | (256, 256, 3, 3) | | | deconv2.conv2d.bias | (256,) | | | deconv3.conv2d | 0.771K | 12.583M | | deconv3.conv2d.weight | (3, 256, 1, 1) | | | deconv3.conv2d.bias | (3,) | |

The pypi package https://pypi.org/project/fvcore/ has not been updated for a while. This is because CI creates package with invalid name, e.g. https://app.circleci.com/pipelines/github/facebookresearch/fvcore/920/workflows/651c790b-758d-4fa0-9471-3317d0280868/jobs/3059/parallel-runs/0/steps/0-106 has fvcore-0.1.5.post20220106.post20220106.post20220106/.

The name is now normal with this commit, shown in https://app.circleci.com/pipelines/github/facebookresearch/fvcore/921/workflows/cb411117-6c4a-48b8-b5d0-486322b0d952/jobs/3064.

The gpu tests are failing due to pre-existing docker error (there is an internal task on it).

Summary: caused https://github.com/facebookresearch/fvcore/issues/87 when trying to migrate fvcore to ao, this fixes by gating before importing

Differential Revision: D31706627

Directly iterating over multipart geometries is being removed in an upcoming version of Shapely and triggers excessive warnings with current versions of Shapely. Switching to iterating over the .geoms field as suggested by the migration guide: https://shapely.readthedocs.io/en/stable/migration.html#multi-part-geometries-will-no-longer-be-sequences-length-iterable-indexable

Summary: Make _BASE_ works for a list of files, loaded in natural order (latter base will overwrite the earlier ones)

Differential Revision: D31842473

[12/11 22:09:15 adet.trainer]: Starting training from iteration 0 Traceback (most recent call last): File "C:\Users\lin\Desktop\Cross_custom_dataset\tools\train_net.py", line 235, in args=(args, ), File "C:\Users\lin\Desktop\Cross_custom_dataset\detectron2\detectron2\engine\launch.py", line 82, in launch main_func(*args) File "C:\Users\lin\Desktop\Cross_custom_dataset\tools\train_net.py", line 223, in main return trainer.train() File "C:\Users\lin\Desktop\Cross_custom_dataset\tools\train_net.py", line 97, in train self.train_loop(self.start_iter, self.max_iter) File "C:\Users\lin\Desktop\Cross_custom_dataset\tools\train_net.py", line 87, in train_loop self.run_step() File "C:\Users\lin\Desktop\Cross_custom_dataset\detectron2\detectron2\engine\defaults.py", line 495, in run_step self._trainer.run_step() File "C:\Users\lin\Desktop\Cross_custom_dataset\detectron2\detectron2\engine\train_loop.py", line 274, in run_step loss_dict = self.model(data) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\lin\Desktop\Cross_custom_dataset\adet\modeling\condinst\crossvis.py", line 140, in forward return self.forward_train(batched_inputs) File "C:\Users\lin\Desktop\Cross_custom_dataset\adet\modeling\condinst\crossvis.py", line 148, in forward_train print("FLOPs: ", flops.total()) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 248, in total stats = self._analyze() File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 551, in _analyze graph = _get_scoped_trace_graph(self._model, self._inputs, self._aliases) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 176, in _get_scoped_trace_graph graph, _ = _get_trace_graph(module, inputs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\jit_trace.py", line 1139, in _get_trace_graph outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\jit_trace.py", line 130, in forward self._force_outplace, File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\jit_trace.py", line 116, in wrapper outs.append(self.inner(*trace_inputs)) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 887, in _call_impl result = self._slow_forward(*input, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 860, in _slow_forward result = self.forward(*input, **kwargs) File "C:\Users\lin\Desktop\Cross_custom_dataset\adet\modeling\condinst\crossvis.py", line 140, in forward return self.forward_train(batched_inputs) File "C:\Users\lin\Desktop\Cross_custom_dataset\adet\modeling\condinst\crossvis.py", line 148, in forward_train print("FLOPs: ", flops.total()) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 248, in total stats = self._analyze() File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 551, in _analyze graph = _get_scoped_trace_graph(self._model, self._inputs, self._aliases) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\fvcore\nn\jit_analysis.py", line 176, in _get_scoped_trace_graph graph, _ = _get_trace_graph(module, inputs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\jit_trace.py", line 1139, in _get_trace_graph outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 887, in _call_impl result = self._slow_forward(*input, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\nn\modules\module.py", line 860, in _slow_forward result = self.forward(*input, **kwargs) File "C:\Users\lin\anaconda3\envs\cross\lib\site-packages\torch\jit_trace.py", line 130, in forward self._force_outplace, RuntimeError: Tracing can't be nested

I tested flop counting for DeformConv2d and I am getting 0 for flops. The code to reproduce is as follows. Please kindly advise how to get the correct FLOPs.

import torch
from fvcore.nn import FlopCountAnalysis, flop_count_table
from torchvision.ops import DeformConv2d

kernel_size = 3
conv = DeformConv2d(64, 64, kernel_size=kernel_size, padding=1)
x = torch.randn(1, 64, 64, 64)
offset = torch.randn(1, 2 * kernel_size * kernel_size, 64, 64)

flops = FlopCountAnalysis(conv, (x, offset))
print(flop_count_table(flops, max_depth=1))

| module   | #parameters or shape   | #flops   |
|:---------|:-----------------------|:---------|
| model    | 36.928K                | 0        |
|  weight  |  (64, 64, 3, 3)        |          |
|  bias    |  (64,)                 |          |

As per title, baddbmm is not supported and is quite common

Can submit PR with below unless any issues/concerns...

def baddbmm_flop_jit(inputs: List[Any], outputs: List[Any]) -> Number:
    """
    Count flops for fully connected layers.
    """
    input_shapes = [get_shape(v) for v in inputs[1:3]]
    n, c, t = input_shapes[0]
    d = input_shapes[-1][-1]
    flop = n * c * t * d
    return flop

Fixes deprecation warning as per https://github.com/facebookresearch/fvcore/issues/103.

Linted/blacked + tests pass locally, although I don't believe there is a test case that covers this anyway (the existing transform works, but it will print a warning every time).

Hello, thank you so much for your work.

I'm trying to use the FlopCountAnalysis class but I'm not able to free the GPU memory used.

As a minimal example, without the FlopCountAnalysis I can do something like:

from transformers import AutoModel
import torch

model = AutoModel.from_pretrained('dccuchile/bert-base-spanish-wwm-uncased')
query = torch.randint(low=0, high=20, size=(8, 16))

print(torch.cuda.memory_allocated())

model.to("cuda:0")
query = query.to("cuda:0")

print(torch.cuda.memory_allocated())

del model
del query

print(torch.cuda.memory_allocated())

And that prints "0", "439937024", "0".

When using the FlopCountAnalysis:

from transformers import AutoModel
import torch
from fvcore.nn import FlopCountAnalysis

model = AutoModel.from_pretrained('dccuchile/bert-base-spanish-wwm-uncased')
query = torch.randint(low=0, high=20, size=(8, 16))

print(torch.cuda.memory_allocated())

model.to("cuda:0")
query = query.to("cuda:0")

print(torch.cuda.memory_allocated())

counter = FlopCountAnalysis(model, inputs=query)
total = counter.total()

print(torch.cuda.memory_allocated())

del model
del query
del counter
del total

print(torch.cuda.memory_allocated())

It shows "0", "439937024", "530033664", "530033664". I expect the final memory allocated to be 0 again.

I also tried with:

gc.collect()
torch.cuda.empty_cache()

at the end, but the result was the same.

Is there a proper way to free the memory?

Thank you.