Thanks for sharing your great work. I am sorry that I have a bug when I use python ./tools/train_net.py --num-gpus 1 --config-file ./configs/yolof_R_50_C5_1x.yaml

Bug log below as :

[03/26 07:38:03 d2.data.build]: Using training sampler TrainingSampler [03/26 07:38:03 d2.data.common]: Serializing 117266 elements to byte tensors and concatenating them all ... [03/26 07:38:10 d2.data.common]: Serialized dataset takes 451.21 MiB [03/26 07:38:15 fvcore.common.checkpoint]: Loading checkpoint from detectron2://ImageNetPretrained/MSRA/R-50.pkl Traceback (most recent call last): File "./tools/train_net.py", line 234, in args=(args,), File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/detectron2/engine/launch.py", line 62, in launch main_func(*args) File "./tools/train_net.py", line 215, in main trainer.resume_or_load(resume=args.resume) File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/detectron2/engine/defaults.py", line 353, in resume_or_load checkpoint = self.checkpointer.resume_or_load(self.cfg.MODEL.WEIGHTS, resume=resume) File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/fvcore/common/checkpoint.py", line 215, in resume_or_load return self.load(path, checkpointables=[]) File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/fvcore/common/checkpoint.py", line 140, in load path = self.path_manager.get_local_path(path) File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/iopath/common/file_io.py", line 1100, in get_local_path path, force=force, **kwargs File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/detectron2/utils/file_io.py", line 29, in _get_local_path return PathManager.get_local_path(self.S3_DETECTRON2_PREFIX + name, **kwargs) File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/iopath/common/file_io.py", line 1100, in get_local_path path, force=force, **kwargs File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/iopath/common/file_io.py", line 755, in _get_local_path with file_lock(cached): File "/home/cw/miniconda3/envs/yolof/lib/python3.6/site-packages/iopath/common/file_io.py", line 82, in file_lock return portalocker.Lock(path + ".lock", timeout=3600) # type: ignore AttributeError: module 'portalocker' has no attribute 'Lock'

I woule be grateful if you could give me some advice. Thanks.

My experimental equipment is 3xTitan, and according to the rules of Detectron2, set the learning rate to 0.045. Without modifying any parameters, the resulting map is about 35.6. why?

during training, you get the index of box which iou more than 0.7 and less 0.10. however, using pred_class_logits[valid_idxs] to compute loss，to set more than 0.7 sanple are negative .why? i think that boxes greater than 0.7 should be classified accurately. So, may be set these box to get training?

Thank you. I want to know how the performance of this depository is trained ？Is it exactly the same as in the table? If not, can you post the specific performance?

Sir. the problem : there is only one category in my dataset ,so I change the config and run the code . can train it in several iteration sometimes, then meet this error.

Traceback (most recent call last): File "./tools/w_train.py", line 270, in args=(args,), File "/home/wuliang/cvprojects/detectron2/detectron2/engine/launch.py", line 82, in launch main_func(*args) File "./tools/w_train.py", line 257, in main return trainer.train() File "/home/wuliang/cvprojects/detectron2/detectron2/engine/defaults.py", line 485, in train super().train(self.start_iter, self.max_iter) File "/home/wuliang/cvprojects/detectron2/detectron2/engine/train_loop.py", line 149, in train self.run_step() File "/home/wuliang/cvprojects/detectron2/detectron2/engine/defaults.py", line 495, in run_step self._trainer.run_step() File "/home/wuliang/cvprojects/detectron2/detectron2/engine/train_loop.py", line 273, in run_step loss_dict = self.model(data) File "/home/wuliang/anaconda3/envs/pyt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/home/wuliang/cvprojects/YOLOF/yolof/modeling/yolof.py", line 295, in forward pred_logits, pred_anchor_deltas) File "/home/wuliang/cvprojects/YOLOF/yolof/modeling/yolof.py", line 404, in losses pred_class_logits[valid_idxs], RuntimeError: CUDA error: device-side assert triggered

Dear author,I have had a problem when I saw the code:

where you aren't define the cfg.SOLVER.MOMENTUM in the configs.Appreciate!

File "/home/zzf/miniconda3/envs/torch1.7.1/lib/python3.7/site-packages/fvcore-0.1.3.post20210317-py3.7.egg/fvcore/nn/giou_loss.py", line 32, in giou_loss AssertionError: bad box: x1 larger than x2

Hello, dear author! When I train YOLOF with my own dataset, an error occurs. The error is as follows: File "G:\Anaconda3\envs\xyy_detectron2\lib\site-packages\fvcore\nn\giou_loss.py", line 32, in giou_loss assert (x2 >= x1).all(), "bad box: x1 larger than x2" AssertionError: bad box: x1 larger than x2 Always run a few epochs will report such an error, try to comment out the assertion, the loss will be nan. How to solve this problem? I would be very grateful if you could help me.

[04/14 16:15:25 d2.engine.hooks]: Total training time: 0:00:24 (0:00:00 on hooks) [04/14 16:15:25 d2.utils.events]: iter: 0 lr: N/A max_mem: 7597M Traceback (most recent call last): File "./tools/train_net.py", line 234, in args=(args,), File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/launch.py", line 79, in launch daemon=False, File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes while not context.join(): File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 150, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 1 terminated with the following error: Traceback (most recent call last): File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap fn(i, *args) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/launch.py", line 125, in _distributed_worker main_func(*args) File "/media/ubun/BE5A462D5A45E32F/detectron2/YOLOF/tools/train_net.py", line 221, in main return trainer.train() File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/defaults.py", line 480, in train super().train(self.start_iter, self.max_iter) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/train_loop.py", line 149, in train self.run_step() File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/defaults.py", line 490, in run_step self._trainer.run_step() File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/engine/train_loop.py", line 273, in run_step loss_dict = self.model(data) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 705, in forward output = self.module(*inputs[0], **kwargs[0]) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/media/ubun/BE5A462D5A45E32F/detectron2/YOLOF/yolof/modeling/yolof.py", line 273, in forward features = self.backbone(images.tensor) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/modeling/backbone/resnet.py", line 449, in forward x = stage(x) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/container.py", line 119, in forward input = module(input) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/modeling/backbone/resnet.py", line 201, in forward out = self.conv3(out) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/layers/wrappers.py", line 88, in forward x = self.norm(x) File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/media/ubun/BE5A462D5A45E32F/detectron2/detectron2/layers/batch_norm.py", line 65, in forward eps=self.eps, File "/home/ubun/anaconda3/envs/detectrontwo/lib/python3.6/site-packages/torch/nn/functional.py", line 2150, in batch_norm input, weight, bias, running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled RuntimeError: CUDA out of memory. Tried to allocate 2.04 GiB (GPU 1; 11.78 GiB total capacity; 5.89 GiB already allocated; 751.50 MiB free; 9.00 GiB reserved in total by PyTorch)

In the course of my training, I made this error but it was predicting delta。in the logic, it shouldn't have occurred , whether or not I downloaded the trained weights, I would have thrown this error when I iterated around 400。

Very good job, but I don't know why The Uniform Matching part of the paper and the code are inconsistent?

paper:

1. adopting the k nearest anchor as positive anchors for each ground-truth box
2. set IoU thresholds in Uniform Matching to ignore large IoU (>0:7) negative anchors and small IoU (<0.15) positive anchors

code:

# positive indices when matching predict boxes and gt boxes
        indices = [
            tuple(
                torch.topk(
                    c[i],
                    k=self.match_times,
                    dim=0,
                    largest=False)[1].numpy().tolist()
            )
            for i, c in enumerate(C.split(sizes, -1))
        ]
 # positive indices when matching anchor boxes and gt boxes
        indices1 = [
            tuple(
                torch.topk(
                    c[i],
                    k=self.match_times,
                    dim=0,
                    largest=False)[1].numpy().tolist())
            for i, c in enumerate(C1.split(sizes, -1))]

Looking forward to your answer！

训练时使用X-101-64x4d作为主干网络，下载的预训练权重无法使用，权重文件的下载地址在detectron文件夹中。 但是其他主干网络的预训练权重的下载地址都在detectron2文件夹中，是因为X-101-64x4d的预训练权重和detectron2的环境不匹配吗，请问作者在训练时有遇到过这样的问题吗？

[12/17 21:15:06 d2.data.build]: Using training sampler TrainingSampler [12/17 21:15:06 d2.data.common]: Serializing 5199 elements to byte tensors and concatenating them all ... [12/17 21:15:06 d2.data.common]: Serialized dataset takes 16.94 MiB [12/17 21:15:06 fvcore.common.checkpoint]: [Checkpointer] Loading from catalog://ImageNetPretrained/FAIR/X-101-64x4d ... [12/17 21:15:06 d2.checkpoint.catalog]: Catalog entry catalog://ImageNetPretrained/FAIR/X-101-64x4d points to https://dl.fbaipublicfiles.com/detectron/ImageNetPretrained/FBResNeXt/X-101-64x4d.pkl Traceback (most recent call last): File "train_Rtinanet.py", line 175, in args=(args,), File "D:\paddle\detectron2\detectron2\engine\launch.py", line 82, in launch main_func(*args) File "train_Rtinanet.py", line 158, in main trainer.resume_or_load(resume=args.resume) File "D:\paddle\detectron2\detectron2\engine\defaults.py", line 412, in resume_or_load self.checkpointer.resume_or_load(self.cfg.MODEL.WEIGHTS, resume=resume) File "D:\Anaconda\envs\detectron2\lib\site-packages\fvcore\common\checkpoint.py", line 229, in resume_or_load return self.load(path, checkpointables=[]) File "D:\paddle\detectron2\detectron2\checkpoint\detection_checkpoint.py", line 52, in load ret = super().load(path, *args, **kwargs) File "D:\Anaconda\envs\detectron2\lib\site-packages\fvcore\common\checkpoint.py", line 156, in load incompatible = self._load_model(checkpoint) File "D:\paddle\detectron2\detectron2\checkpoint\detection_checkpoint.py", line 95, in _load_model self._convert_ndarray_to_tensor(checkpoint["model"]) File "D:\Anaconda\envs\detectron2\lib\site-packages\fvcore\common\checkpoint.py", line 372, in _convert_ndarray_to_tensor "Unsupported type found in checkpoint! {}: {}".format(k, type(v)) ValueError: Unsupported type found in checkpoint! weight_order: <class 'str'>

[07/11 18:24:42 d2.engine.train_loop]: Starting training from iteration 0 [07/11 18:24:52 d2.utils.events]: eta: 7:53:07 iter: 19 total_loss: 1.522 loss_cls: 0.2083 loss_box_reg: 0.0001553 loss_mask: 0.6411 loss_rpn_cls: 0.6729 loss_rpn_loc: 0.02641 time: 0.4052 data_time: 0.0427 lr: 0.00019981 max_mem: 6824M [07/11 18:25:00 d2.utils.events]: eta: 7:58:34 iter: 39 total_loss: 0.826 loss_cls: 0.02481 loss_box_reg: 0.001338 loss_mask: 0.3514 loss_rpn_cls: 0.3847 loss_rpn_loc: 0.0112 time: 0.4081 data_time: 0.0091 lr: 0.00039961 max_mem: 7054M [07/11 18:25:09 d2.utils.events]: eta: 8:03:20 iter: 59 total_loss: 0.4657 loss_cls: 0.0525 loss_box_reg: 0.00528 loss_mask: 0.2328 loss_rpn_cls: 0.1254 loss_rpn_loc: 0.01294 time: 0.4081 data_time: 0.0062 lr: 0.00059941 max_mem: 7054M [07/11 18:25:17 d2.utils.events]: eta: 8:10:51 iter: 79 total_loss: 0.5412 loss_cls: 0.1219 loss_box_reg: 0.01392 loss_mask: 0.1982 loss_rpn_cls: 0.1097 loss_rpn_loc: 0.04281 time: 0.4142 data_time: 0.0080 lr: 0.00079921 max_mem: 8390M [07/11 18:25:26 d2.utils.events]: eta: 8:12:44 iter: 99 total_loss: 0.3733 loss_cls: 0.09589 loss_box_reg: 0.03402 loss_mask: 0.1765 loss_rpn_cls: 0.07032 loss_rpn_loc: 0.02893 time: 0.4211 data_time: 0.0077 lr: 0.00099901 max_mem: 10375M [07/11 18:25:35 d2.utils.events]: eta: 8:17:41 iter: 119 total_loss: 0.3755 loss_cls: 0.08576 loss_box_reg: 0.03418 loss_mask: 0.1741 loss_rpn_cls: 0.0485 loss_rpn_loc: 0.02191 time: 0.4233 data_time: 0.0083 lr: 0.0011988 max_mem: 10375M [07/11 18:25:44 d2.utils.events]: eta: 8:24:34 iter: 139 total_loss: 0.3946 loss_cls: 0.09633 loss_box_reg: 0.0391 loss_mask: 0.1771 loss_rpn_cls: 0.04919 loss_rpn_loc: 0.04056 time: 0.4265 data_time: 0.0084 lr: 0.0013986 max_mem: 10375M [07/11 18:25:53 d2.utils.events]: eta: 8:28:27 iter: 159 total_loss: 0.4053 loss_cls: 0.08783 loss_box_reg: 0.03864 loss_mask: 0.1736 loss_rpn_cls: 0.04172 loss_rpn_loc: 0.04759 time: 0.4287 data_time: 0.0093 lr: 0.0015984 max_mem: 10375M [07/11 18:26:02 d2.utils.events]: eta: 8:28:45 iter: 179 total_loss: 0.4056 loss_cls: 0.1044 loss_box_reg: 0.03753 loss_mask: 0.1796 loss_rpn_cls: 0.03264 loss_rpn_loc: 0.0305 time: 0.4302 data_time: 0.0074 lr: 0.0017982 max_mem: 10375M [07/11 18:26:11 d2.utils.events]: eta: 8:28:40 iter: 199 total_loss: 0.3904 loss_cls: 0.08258 loss_box_reg: 0.02545 loss_mask: 0.1929 loss_rpn_cls: 0.03777 loss_rpn_loc: 0.02238 time: 0.4302 data_time: 0.0071 lr: 0.001998 max_mem: 10375M [07/11 18:26:20 d2.utils.events]: eta: 8:30:04 iter: 219 total_loss: 0.3945 loss_cls: 0.09928 loss_box_reg: 0.02967 loss_mask: 0.1953 loss_rpn_cls: 0.0315 loss_rpn_loc: 0.03111 time: 0.4312 data_time: 0.0092 lr: 0.0021978 max_mem: 10375M [07/11 18:26:28 d2.utils.events]: eta: 8:29:56 iter: 239 total_loss: 0.3698 loss_cls: 0.08268 loss_box_reg: 0.02776 loss_mask: 0.1708 loss_rpn_cls: 0.0224 loss_rpn_loc: 0.04258 time: 0.4310 data_time: 0.0067 lr: 0.0023976 max_mem: 10375M [07/11 18:26:37 d2.utils.events]: eta: 8:30:41 iter: 259 total_loss: 0.3101 loss_cls: 0.05248 loss_box_reg: 0.01284 loss_mask: 0.1519 loss_rpn_cls: 0.01409 loss_rpn_loc: 0.02101 time: 0.4315 data_time: 0.0078 lr: 0.0025974 max_mem: 10375M [07/11 18:26:46 d2.utils.events]: eta: 8:29:39 iter: 279 total_loss: 0.3361 loss_cls: 0.07323 loss_box_reg: 0.01987 loss_mask: 0.1451 loss_rpn_cls: 0.02021 loss_rpn_loc: 0.0349 time: 0.4326 data_time: 0.0072 lr: 0.0027972 max_mem: 10375M [07/11 18:26:55 d2.utils.events]: eta: 8:28:24 iter: 299 total_loss: 0.2654 loss_cls: 0.06387 loss_box_reg: 0.01594 loss_mask: 0.1451 loss_rpn_cls: 0.01477 loss_rpn_loc: 0.01315 time: 0.4324 data_time: 0.0065 lr: 0.002997 max_mem: 10375M [07/11 18:27:04 d2.utils.events]: eta: 8:30:20 iter: 319 total_loss: 0.3898 loss_cls: 0.08069 loss_box_reg: 0.02259 loss_mask: 0.1694 loss_rpn_cls: 0.02082 loss_rpn_loc: 0.03517 time: 0.4336 data_time: 0.0093 lr: 0.0031968 max_mem: 10375M [07/11 18:27:13 d2.utils.events]: eta: 8:30:33 iter: 339 total_loss: 0.3345 loss_cls: 0.08011 loss_box_reg: 0.02296 loss_mask: 0.161 loss_rpn_cls: 0.0221 loss_rpn_loc: 0.04661 time: 0.4344 data_time: 0.0076 lr: 0.0033966 max_mem: 10375M [07/11 18:27:22 d2.utils.events]: eta: 8:30:21 iter: 359 total_loss: 0.2744 loss_cls: 0.0485 loss_box_reg: 0.01726 loss_mask: 0.1293 loss_rpn_cls: 0.01705 loss_rpn_loc: 0.03203 time: 0.4351 data_time: 0.0077 lr: 0.0035964 max_mem: 10375M [07/11 18:27:31 d2.utils.events]: eta: 8:30:16 iter: 379 total_loss: 0.2558 loss_cls: 0.06494 loss_box_reg: 0.01695 loss_mask: 0.1368 loss_rpn_cls: 0.008527 loss_rpn_loc: 0.01464 time: 0.4352 data_time: 0.0068 lr: 0.0037962 max_mem: 10375M [07/11 18:27:40 d2.utils.events]: eta: 8:30:04 iter: 399 total_loss: 0.246 loss_cls: 0.04827 loss_box_reg: 0.01576 loss_mask: 0.1197 loss_rpn_cls: 0.01373 loss_rpn_loc: 0.01939 time: 0.4349 data_time: 0.0070 lr: 0.003996 max_mem: 10375M [07/11 18:27:48 d2.utils.events]: eta: 8:29:17 iter: 419 total_loss: 0.2877 loss_cls: 0.06765 loss_box_reg: 0.01713 loss_mask: 0.156 loss_rpn_cls: 0.01482 loss_rpn_loc: 0.0207 time: 0.4345 data_time: 0.0082 lr: 0.0041958 max_mem: 10375M [07/11 18:27:57 d2.utils.events]: eta: 8:29:30 iter: 439 total_loss: 0.4209 loss_cls: 0.1212 loss_box_reg: 0.02121 loss_mask: 0.1896 loss_rpn_cls: 0.01865 loss_rpn_loc: 0.03847 time: 0.4343 data_time: 0.0088 lr: 0.0043956 max_mem: 10375M [07/11 18:28:06 d2.utils.events]: eta: 8:29:38 iter: 459 total_loss: 0.4073 loss_cls: 0.1058 loss_box_reg: 0.02056 loss_mask: 0.163 loss_rpn_cls: 0.02908 loss_rpn_loc: 0.04771 time: 0.4348 data_time: 0.0089 lr: 0.0045954 max_mem: 10375M [07/11 18:28:15 d2.utils.events]: eta: 8:30:15 iter: 479 total_loss: 0.3359 loss_cls: 0.08171 loss_box_reg: 0.0161 loss_mask: 0.1719 loss_rpn_cls: 0.02255 loss_rpn_loc: 0.03287 time: 0.4353 data_time: 0.0076 lr: 0.0047952 max_mem: 10375M [07/11 18:28:24 d2.utils.events]: eta: 8:31:50 iter: 499 total_loss: 0.3511 loss_cls: 0.09287 loss_box_reg: 0.02037 loss_mask: 0.1579 loss_rpn_cls: 0.009786 loss_rpn_loc: 0.03596 time: 0.4361 data_time: 0.0082 lr: 0.004995 max_mem: 10375M [07/11 18:28:33 d2.utils.events]: eta: 8:31:51 iter: 519 total_loss: 0.3224 loss_cls: 0.08946 loss_box_reg: 0.02361 loss_mask: 0.1484 loss_rpn_cls: 0.0141 loss_rpn_loc: 0.03703 time: 0.4363 data_time: 0.0084 lr: 0.0051948 max_mem: 10375M [07/11 18:28:42 d2.utils.events]: eta: 8:33:04 iter: 539 total_loss: 0.3617 loss_cls: 0.06344 loss_box_reg: 0.01625 loss_mask: 0.2091 loss_rpn_cls: 0.01894 loss_rpn_loc: 0.02714 time: 0.4365 data_time: 0.0091 lr: 0.0053946 max_mem: 10375M [07/11 18:28:51 d2.utils.events]: eta: 8:33:12 iter: 559 total_loss: 0.3383 loss_cls: 0.09253 loss_box_reg: 0.01976 loss_mask: 0.1941 loss_rpn_cls: 0.0146 loss_rpn_loc: 0.02182 time: 0.4367 data_time: 0.0078 lr: 0.0055944 max_mem: 10375M [07/11 18:28:59 d2.utils.events]: eta: 8:33:44 iter: 579 total_loss: 0.3444 loss_cls: 0.08604 loss_box_reg: 0.01819 loss_mask: 0.1638 loss_rpn_cls: 0.01426 loss_rpn_loc: 0.02466 time: 0.4369 data_time: 0.0084 lr: 0.0057942 max_mem: 10375M [07/11 18:29:08 d2.utils.events]: eta: 8:32:55 iter: 599 total_loss: 0.3361 loss_cls: 0.07452 loss_box_reg: 0.01833 loss_mask: 0.148 loss_rpn_cls: 0.02424 loss_rpn_loc: 0.03891 time: 0.4367 data_time: 0.0082 lr: 0.005994 max_mem: 10375M [07/11 18:29:17 d2.utils.events]: eta: 8:32:46 iter: 619 total_loss: 0.3567 loss_cls: 0.0879 loss_box_reg: 0.01742 loss_mask: 0.1598 loss_rpn_cls: 0.009541 loss_rpn_loc: 0.01607 time: 0.4367 data_time: 0.0081 lr: 0.0061938 max_mem: 10375M [07/11 18:29:26 d2.utils.events]: eta: 8:32:22 iter: 639 total_loss: 0.2534 loss_cls: 0.07315 loss_box_reg: 0.01847 loss_mask: 0.1428 loss_rpn_cls: 0.006772 loss_rpn_loc: 0.01518 time: 0.4364 data_time: 0.0067 lr: 0.0063936 max_mem: 10375M [07/11 18:29:34 d2.utils.events]: eta: 8:32:10 iter: 659 total_loss: 0.3182 loss_cls: 0.06682 loss_box_reg: 0.01769 loss_mask: 0.178 loss_rpn_cls: 0.007947 loss_rpn_loc: 0.02726 time: 0.4364 data_time: 0.0077 lr: 0.0065934 max_mem: 10375M [07/11 18:29:43 d2.utils.events]: eta: 8:32:20 iter: 679 total_loss: 0.3215 loss_cls: 0.07152 loss_box_reg: 0.01705 loss_mask: 0.1694 loss_rpn_cls: 0.01041 loss_rpn_loc: 0.02417 time: 0.4365 data_time: 0.0086 lr: 0.0067932 max_mem: 10375M [07/11 18:29:52 d2.utils.events]: eta: 8:32:46 iter: 699 total_loss: 0.2565 loss_cls: 0.06282 loss_box_reg: 0.02091 loss_mask: 0.1476 loss_rpn_cls: 0.006757 loss_rpn_loc: 0.01335 time: 0.4368 data_time: 0.0075 lr: 0.006993 max_mem: 10375M [07/11 18:30:01 d2.utils.events]: eta: 8:32:03 iter: 719 total_loss: 0.2769 loss_cls: 0.07035 loss_box_reg: 0.01751 loss_mask: 0.1437 loss_rpn_cls: 0.009603 loss_rpn_loc: 0.02026 time: 0.4365 data_time: 0.0072 lr: 0.0071928 max_mem: 10375M [07/11 18:30:10 d2.utils.events]: eta: 8:32:29 iter: 739 total_loss: 0.3333 loss_cls: 0.09517 loss_box_reg: 0.0208 loss_mask: 0.1501 loss_rpn_cls: 0.01084 loss_rpn_loc: 0.03651 time: 0.4370 data_time: 0.0096 lr: 0.0073926 max_mem: 10375M [07/11 18:30:19 d2.utils.events]: eta: 8:32:32 iter: 759 total_loss: 0.3375 loss_cls: 0.08592 loss_box_reg: 0.02321 loss_mask: 0.1568 loss_rpn_cls: 0.008646 loss_rpn_loc: 0.0306 time: 0.4375 data_time: 0.0076 lr: 0.0075924 max_mem: 10375M [07/11 18:30:28 d2.utils.events]: eta: 8:32:09 iter: 779 total_loss: 0.3126 loss_cls: 0.06175 loss_box_reg: 0.0145 loss_mask: 0.1849 loss_rpn_cls: 0.01163 loss_rpn_loc: 0.01285 time: 0.4372 data_time: 0.0082 lr: 0.0077922 max_mem: 10375M [07/11 18:30:37 d2.utils.events]: eta: 8:32:03 iter: 799 total_loss: 0.261 loss_cls: 0.06443 loss_box_reg: 0.01786 loss_mask: 0.1488 loss_rpn_cls: 0.01037 loss_rpn_loc: 0.02081 time: 0.4375 data_time: 0.0085 lr: 0.007992 max_mem: 10375M [07/11 18:30:45 d2.utils.events]: eta: 8:31:52 iter: 819 total_loss: 0.2574 loss_cls: 0.05869 loss_box_reg: 0.01517 loss_mask: 0.1326 loss_rpn_cls: 0.008631 loss_rpn_loc: 0.02182 time: 0.4372 data_time: 0.0073 lr: 0.0081918 max_mem: 10375M [07/11 18:30:55 d2.utils.events]: eta: 8:31:43 iter: 839 total_loss: 0.333 loss_cls: 0.08468 loss_box_reg: 0.02047 loss_mask: 0.1637 loss_rpn_cls: 0.01798 loss_rpn_loc: 0.03197 time: 0.4375 data_time: 0.0077 lr: 0.0083916 max_mem: 10375M [07/11 18:31:03 d2.utils.events]: eta: 8:31:13 iter: 859 total_loss: 0.277 loss_cls: 0.07466 loss_box_reg: 0.01636 loss_mask: 0.1659 loss_rpn_cls: 0.006309 loss_rpn_loc: 0.0188 time: 0.4370 data_time: 0.0075 lr: 0.0085914 max_mem: 10375M [07/11 18:31:12 d2.utils.events]: eta: 8:31:40 iter: 879 total_loss: 0.3506 loss_cls: 0.09326 loss_box_reg: 0.02224 loss_mask: 0.1705 loss_rpn_cls: 0.01516 loss_rpn_loc: 0.03855 time: 0.4375 data_time: 0.0082 lr: 0.0087912 max_mem: 10375M [07/11 18:31:21 d2.utils.events]: eta: 8:31:31 iter: 899 total_loss: 0.2598 loss_cls: 0.07439 loss_box_reg: 0.01754 loss_mask: 0.1518 loss_rpn_cls: 0.005731 loss_rpn_loc: 0.01866 time: 0.4375 data_time: 0.0079 lr: 0.008991 max_mem: 10375M [07/11 18:31:30 d2.utils.events]: eta: 8:31:23 iter: 919 total_loss: 0.2764 loss_cls: 0.08773 loss_box_reg: 0.01714 loss_mask: 0.1435 loss_rpn_cls: 0.006121 loss_rpn_loc: 0.01353 time: 0.4374 data_time: 0.0077 lr: 0.0091908 max_mem: 10375M [07/11 18:31:39 d2.utils.events]: eta: 8:31:23 iter: 939 total_loss: 0.264 loss_cls: 0.08377 loss_box_reg: 0.01873 loss_mask: 0.1317 loss_rpn_cls: 0.008426 loss_rpn_loc: 0.02521 time: 0.4376 data_time: 0.0067 lr: 0.0093906 max_mem: 10375M [07/11 18:31:47 d2.utils.events]: eta: 8:31:05 iter: 959 total_loss: 0.249 loss_cls: 0.04743 loss_box_reg: 0.008316 loss_mask: 0.1123 loss_rpn_cls: 0.005139 loss_rpn_loc: 0.006432 time: 0.4374 data_time: 0.0071 lr: 0.0095904 max_mem: 10375M [07/11 18:31:56 d2.utils.events]: eta: 8:31:19 iter: 979 total_loss: 0.3152 loss_cls: 0.1002 loss_box_reg: 0.01721 loss_mask: 0.1611 loss_rpn_cls: 0.01409 loss_rpn_loc: 0.02712 time: 0.4376 data_time: 0.0092 lr: 0.0097902 max_mem: 10375M [07/11 18:32:05 d2.utils.events]: eta: 8:31:14 iter: 999 total_loss: 0.3033 loss_cls: 0.1042 loss_box_reg: 0.02027 loss_mask: 0.153 loss_rpn_cls: 0.01101 loss_rpn_loc: 0.02101 time: 0.4375 data_time: 0.0085 lr: 0.00999 max_mem: 10375M [07/11 18:32:14 d2.utils.events]: eta: 8:31:48 iter: 1019 total_loss: 0.3161 loss_cls: 0.08504 loss_box_reg: 0.02013 loss_mask: 0.1508 loss_rpn_cls: 0.01067 loss_rpn_loc: 0.03305 time: 0.4376 data_time: 0.0087 lr: 0.01 max_mem: 10375M [07/11 18:32:23 d2.utils.events]: eta: 8:33:14 iter: 1039 total_loss: 0.2638 loss_cls: 0.05909 loss_box_reg: 0.01622 loss_mask: 0.1384 loss_rpn_cls: 0.008573 loss_rpn_loc: 0.0123 time: 0.4377 data_time: 0.0087 lr: 0.01 max_mem: 10375M [07/11 18:32:31 d2.utils.events]: eta: 8:33:05 iter: 1059 total_loss: 0.2521 loss_cls: 0.06265 loss_box_reg: 0.0175 loss_mask: 0.1267 loss_rpn_cls: 0.006054 loss_rpn_loc: 0.0113 time: 0.4373 data_time: 0.0076 lr: 0.01 max_mem: 10375M [07/11 18:32:40 d2.utils.events]: eta: 8:34:00 iter: 1079 total_loss: 0.3011 loss_cls: 0.09204 loss_box_reg: 0.02271 loss_mask: 0.1404 loss_rpn_cls: 0.006699 loss_rpn_loc: 0.02894 time: 0.4375 data_time: 0.0080 lr: 0.01 max_mem: 10375M [07/11 18:32:49 d2.utils.events]: eta: 8:33:59 iter: 1099 total_loss: 0.3083 loss_cls: 0.09003 loss_box_reg: 0.01742 loss_mask: 0.1526 loss_rpn_cls: 0.01039 loss_rpn_loc: 0.02548 time: 0.4377 data_time: 0.0084 lr: 0.01 max_mem: 10375M [07/11 18:32:58 d2.utils.events]: eta: 8:33:51 iter: 1119 total_loss: 0.3485 loss_cls: 0.106 loss_box_reg: 0.02342 loss_mask: 0.1609 loss_rpn_cls: 0.008596 loss_rpn_loc: 0.02881 time: 0.4379 data_time: 0.0084 lr: 0.01 max_mem: 10375M [07/11 18:33:07 d2.utils.events]: eta: 8:33:28 iter: 1139 total_loss: 0.2774 loss_cls: 0.07644 loss_box_reg: 0.02027 loss_mask: 0.1465 loss_rpn_cls: 0.01067 loss_rpn_loc: 0.03347 time: 0.4379 data_time: 0.0093 lr: 0.01 max_mem: 10375M [07/11 18:33:16 d2.utils.events]: eta: 8:32:37 iter: 1159 total_loss: 0.2716 loss_cls: 0.07568 loss_box_reg: 0.02224 loss_mask: 0.1451 loss_rpn_cls: 0.005922 loss_rpn_loc: 0.01752 time: 0.4379 data_time: 0.0085 lr: 0.01 max_mem: 10375M [07/11 18:33:24 d2.utils.events]: eta: 8:32:29 iter: 1179 total_loss: 0.23 loss_cls: 0.07258 loss_box_reg: 0.01687 loss_mask: 0.125 loss_rpn_cls: 0.004398 loss_rpn_loc: 0.01229 time: 0.4378 data_time: 0.0071 lr: 0.01 max_mem: 10375M [07/11 18:33:33 d2.utils.events]: eta: 8:32:32 iter: 1199 total_loss: 0.3609 loss_cls: 0.08158 loss_box_reg: 0.02385 loss_mask: 0.1824 loss_rpn_cls: 0.01222 loss_rpn_loc: 0.03337 time: 0.4380 data_time: 0.0081 lr: 0.01 max_mem: 10375M [07/11 18:33:42 d2.utils.events]: eta: 8:32:01 iter: 1219 total_loss: 0.2507 loss_cls: 0.0415 loss_box_reg: 0.009657 loss_mask: 0.1466 loss_rpn_cls: 0.01578 loss_rpn_loc: 0.03665 time: 0.4378 data_time: 0.0085 lr: 0.01 max_mem: 10375M [07/11 18:33:52 d2.utils.events]: eta: 8:32:51 iter: 1239 total_loss: 0.3217 loss_cls: 0.0634 loss_box_reg: 0.01922 loss_mask: 0.1806 loss_rpn_cls: 0.01585 loss_rpn_loc: 0.0405 time: 0.4382 data_time: 0.0086 lr: 0.01 max_mem: 10824M [07/11 18:34:00 d2.utils.events]: eta: 8:32:42 iter: 1259 total_loss: 0.2819 loss_cls: 0.05084 loss_box_reg: 0.0185 loss_mask: 0.1524 loss_rpn_cls: 0.01591 loss_rpn_loc: 0.02505 time: 0.4381 data_time: 0.0075 lr: 0.01 max_mem: 10824M [07/11 18:34:09 d2.utils.events]: eta: 8:32:48 iter: 1279 total_loss: 0.3119 loss_cls: 0.05793 loss_box_reg: 0.01852 loss_mask: 0.1548 loss_rpn_cls: 0.01206 loss_rpn_loc: 0.03262 time: 0.4382 data_time: 0.0074 lr: 0.01 max_mem: 10824M [07/11 18:34:18 d2.utils.events]: eta: 8:32:56 iter: 1299 total_loss: 0.2717 loss_cls: 0.06279 loss_box_reg: 0.01653 loss_mask: 0.129 loss_rpn_cls: 0.01387 loss_rpn_loc: 0.02503 time: 0.4381 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:34:27 d2.utils.events]: eta: 8:32:24 iter: 1319 total_loss: 0.3009 loss_cls: 0.08376 loss_box_reg: 0.01819 loss_mask: 0.1446 loss_rpn_cls: 0.00914 loss_rpn_loc: 0.01798 time: 0.4379 data_time: 0.0077 lr: 0.01 max_mem: 10824M [07/11 18:34:36 d2.utils.events]: eta: 8:32:26 iter: 1339 total_loss: 0.2284 loss_cls: 0.06878 loss_box_reg: 0.01575 loss_mask: 0.1159 loss_rpn_cls: 0.01002 loss_rpn_loc: 0.02297 time: 0.4381 data_time: 0.0086 lr: 0.01 max_mem: 10824M [07/11 18:34:45 d2.utils.events]: eta: 8:32:33 iter: 1359 total_loss: 0.2712 loss_cls: 0.06552 loss_box_reg: 0.01762 loss_mask: 0.1461 loss_rpn_cls: 0.008251 loss_rpn_loc: 0.02143 time: 0.4383 data_time: 0.0090 lr: 0.01 max_mem: 10824M [07/11 18:34:53 d2.utils.events]: eta: 8:32:08 iter: 1379 total_loss: 0.242 loss_cls: 0.05389 loss_box_reg: 0.01578 loss_mask: 0.1311 loss_rpn_cls: 0.004269 loss_rpn_loc: 0.007739 time: 0.4381 data_time: 0.0070 lr: 0.01 max_mem: 10824M [07/11 18:35:02 d2.utils.events]: eta: 8:32:54 iter: 1399 total_loss: 0.2266 loss_cls: 0.05687 loss_box_reg: 0.01922 loss_mask: 0.1227 loss_rpn_cls: 0.006927 loss_rpn_loc: 0.0143 time: 0.4382 data_time: 0.0069 lr: 0.01 max_mem: 10824M [07/11 18:35:11 d2.utils.events]: eta: 8:32:58 iter: 1419 total_loss: 0.305 loss_cls: 0.08086 loss_box_reg: 0.02218 loss_mask: 0.1419 loss_rpn_cls: 0.006573 loss_rpn_loc: 0.01774 time: 0.4382 data_time: 0.0083 lr: 0.01 max_mem: 10824M [07/11 18:35:20 d2.utils.events]: eta: 8:32:48 iter: 1439 total_loss: 0.3102 loss_cls: 0.08404 loss_box_reg: 0.02221 loss_mask: 0.149 loss_rpn_cls: 0.01271 loss_rpn_loc: 0.04032 time: 0.4383 data_time: 0.0093 lr: 0.01 max_mem: 10824M [07/11 18:35:29 d2.utils.events]: eta: 8:32:28 iter: 1459 total_loss: 0.2952 loss_cls: 0.06519 loss_box_reg: 0.01814 loss_mask: 0.1636 loss_rpn_cls: 0.00677 loss_rpn_loc: 0.02062 time: 0.4381 data_time: 0.0078 lr: 0.01 max_mem: 10824M [07/11 18:35:38 d2.utils.events]: eta: 8:32:31 iter: 1479 total_loss: 0.2856 loss_cls: 0.06728 loss_box_reg: 0.01703 loss_mask: 0.1394 loss_rpn_cls: 0.006174 loss_rpn_loc: 0.02054 time: 0.4385 data_time: 0.0086 lr: 0.01 max_mem: 10824M [07/11 18:35:47 d2.utils.events]: eta: 8:31:51 iter: 1499 total_loss: 0.2848 loss_cls: 0.07849 loss_box_reg: 0.02293 loss_mask: 0.15 loss_rpn_cls: 0.009778 loss_rpn_loc: 0.0196 time: 0.4387 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:35:56 d2.utils.events]: eta: 8:31:17 iter: 1519 total_loss: 0.3311 loss_cls: 0.07853 loss_box_reg: 0.01755 loss_mask: 0.1597 loss_rpn_cls: 0.01422 loss_rpn_loc: 0.02734 time: 0.4387 data_time: 0.0080 lr: 0.01 max_mem: 10824M [07/11 18:36:05 d2.utils.events]: eta: 8:32:08 iter: 1539 total_loss: 0.314 loss_cls: 0.09528 loss_box_reg: 0.02204 loss_mask: 0.1555 loss_rpn_cls: 0.008758 loss_rpn_loc: 0.02828 time: 0.4390 data_time: 0.0095 lr: 0.01 max_mem: 10824M [07/11 18:36:14 d2.utils.events]: eta: 8:32:09 iter: 1559 total_loss: 0.2458 loss_cls: 0.05934 loss_box_reg: 0.01621 loss_mask: 0.1345 loss_rpn_cls: 0.005067 loss_rpn_loc: 0.018 time: 0.4390 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:36:24 d2.utils.events]: eta: 8:32:26 iter: 1579 total_loss: 0.356 loss_cls: 0.1014 loss_box_reg: 0.02792 loss_mask: 0.1477 loss_rpn_cls: 0.007858 loss_rpn_loc: 0.03938 time: 0.4394 data_time: 0.0095 lr: 0.01 max_mem: 10824M [07/11 18:36:33 d2.utils.events]: eta: 8:33:11 iter: 1599 total_loss: 0.2719 loss_cls: 0.08995 loss_box_reg: 0.02081 loss_mask: 0.1336 loss_rpn_cls: 0.007486 loss_rpn_loc: 0.04157 time: 0.4397 data_time: 0.0088 lr: 0.01 max_mem: 10824M [07/11 18:36:42 d2.utils.events]: eta: 8:33:36 iter: 1619 total_loss: 0.2727 loss_cls: 0.06473 loss_box_reg: 0.01508 loss_mask: 0.1268 loss_rpn_cls: 0.00439 loss_rpn_loc: 0.01114 time: 0.4399 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:36:51 d2.utils.events]: eta: 8:33:56 iter: 1639 total_loss: 0.3048 loss_cls: 0.07076 loss_box_reg: 0.01792 loss_mask: 0.1598 loss_rpn_cls: 0.01129 loss_rpn_loc: 0.0204 time: 0.4399 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:37:00 d2.utils.events]: eta: 8:34:01 iter: 1659 total_loss: 0.2019 loss_cls: 0.05851 loss_box_reg: 0.0147 loss_mask: 0.1105 loss_rpn_cls: 0.006341 loss_rpn_loc: 0.01156 time: 0.4400 data_time: 0.0094 lr: 0.01 max_mem: 10824M [07/11 18:37:09 d2.utils.events]: eta: 8:33:38 iter: 1679 total_loss: 0.2913 loss_cls: 0.07293 loss_box_reg: 0.01963 loss_mask: 0.1563 loss_rpn_cls: 0.01009 loss_rpn_loc: 0.01724 time: 0.4401 data_time: 0.0089 lr: 0.01 max_mem: 10824M [07/11 18:37:18 d2.utils.events]: eta: 8:33:19 iter: 1699 total_loss: 0.242 loss_cls: 0.0543 loss_box_reg: 0.01551 loss_mask: 0.1358 loss_rpn_cls: 0.006947 loss_rpn_loc: 0.0167 time: 0.4403 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:37:27 d2.utils.events]: eta: 8:33:28 iter: 1719 total_loss: 0.2673 loss_cls: 0.06433 loss_box_reg: 0.01698 loss_mask: 0.1393 loss_rpn_cls: 0.01027 loss_rpn_loc: 0.026 time: 0.4402 data_time: 0.0089 lr: 0.01 max_mem: 10824M [07/11 18:37:36 d2.utils.events]: eta: 8:33:12 iter: 1739 total_loss: 0.2919 loss_cls: 0.07103 loss_box_reg: 0.02 loss_mask: 0.1496 loss_rpn_cls: 0.006164 loss_rpn_loc: 0.01211 time: 0.4403 data_time: 0.0083 lr: 0.01 max_mem: 10824M [07/11 18:37:45 d2.utils.events]: eta: 8:33:05 iter: 1759 total_loss: 0.2686 loss_cls: 0.04505 loss_box_reg: 0.01351 loss_mask: 0.1634 loss_rpn_cls: 0.009903 loss_rpn_loc: 0.01445 time: 0.4404 data_time: 0.0077 lr: 0.01 max_mem: 10824M [07/11 18:37:54 d2.utils.events]: eta: 8:33:57 iter: 1779 total_loss: 0.2619 loss_cls: 0.06645 loss_box_reg: 0.01745 loss_mask: 0.1382 loss_rpn_cls: 0.006083 loss_rpn_loc: 0.01326 time: 0.4404 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:38:03 d2.utils.events]: eta: 8:33:51 iter: 1799 total_loss: 0.3124 loss_cls: 0.09307 loss_box_reg: 0.02187 loss_mask: 0.1663 loss_rpn_cls: 0.005512 loss_rpn_loc: 0.024 time: 0.4406 data_time: 0.0076 lr: 0.01 max_mem: 10824M [07/11 18:38:12 d2.utils.events]: eta: 8:33:42 iter: 1819 total_loss: 0.2443 loss_cls: 0.05758 loss_box_reg: 0.01611 loss_mask: 0.1355 loss_rpn_cls: 0.004745 loss_rpn_loc: 0.01235 time: 0.4404 data_time: 0.0072 lr: 0.01 max_mem: 10824M [07/11 18:38:21 d2.utils.events]: eta: 8:33:20 iter: 1839 total_loss: 0.2389 loss_cls: 0.05698 loss_box_reg: 0.01671 loss_mask: 0.1417 loss_rpn_cls: 0.008342 loss_rpn_loc: 0.01314 time: 0.4404 data_time: 0.0079 lr: 0.01 max_mem: 10824M [07/11 18:38:30 d2.utils.events]: eta: 8:33:53 iter: 1859 total_loss: 0.2556 loss_cls: 0.07939 loss_box_reg: 0.01739 loss_mask: 0.1392 loss_rpn_cls: 0.009146 loss_rpn_loc: 0.02361 time: 0.4407 data_time: 0.0086 lr: 0.01 max_mem: 10824M [07/11 18:38:39 d2.utils.events]: eta: 8:33:16 iter: 1879 total_loss: 0.268 loss_cls: 0.06665 loss_box_reg: 0.0187 loss_mask: 0.1206 loss_rpn_cls: 0.003925 loss_rpn_loc: 0.01439 time: 0.4408 data_time: 0.0082 lr: 0.01 max_mem: 10824M [07/11 18:38:48 d2.utils.events]: eta: 8:33:08 iter: 1899 total_loss: 0.2089 loss_cls: 0.05096 loss_box_reg: 0.01195 loss_mask: 0.1319 loss_rpn_cls: 0.00131 loss_rpn_loc: 0.005368 time: 0.4407 data_time: 0.0075 lr: 0.01 max_mem: 10824M [07/11 18:38:57 d2.utils.events]: eta: 8:33:05 iter: 1919 total_loss: 0.2939 loss_cls: 0.08017 loss_box_reg: 0.02089 loss_mask: 0.1447 loss_rpn_cls: 0.01021 loss_rpn_loc: 0.02003 time: 0.4408 data_time: 0.0080 lr: 0.01 max_mem: 10824M [07/11 18:39:06 d2.utils.events]: eta: 8:33:25 iter: 1939 total_loss: 0.2455 loss_cls: 0.06749 loss_box_reg: 0.01648 loss_mask: 0.1294 loss_rpn_cls: 0.004726 loss_rpn_loc: 0.02557 time: 0.4409 data_time: 0.0082 lr: 0.01 max_mem: 10824M

*** WARNING: skipped 801942 bytes of output ***

[07/12 01:31:55 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in inference: [RandomRotation(angle=[0.0, 360.0], interp=2)] [07/12 01:31:57 d2.data.common]: Serializing 234 elements to byte tensors and concatenating them all ... [07/12 01:31:57 d2.data.common]: Serialized dataset takes 1.69 MiB WARNING [07/12 01:31:57 d2.evaluation.coco_evaluation]: COCO Evaluator instantiated using config, this is deprecated behavior. Please pass in explicit arguments instead. [07/12 01:31:57 d2.evaluation.coco_evaluation]: Trying to convert 'dataset_test' to COCO format ... [07/12 01:31:57 d2.data.datasets.coco]: Converting annotations of dataset 'dataset_test' to COCO format ...) [07/12 01:31:58 d2.data.datasets.coco]: Converting dataset dicts into COCO format [07/12 01:31:58 d2.data.datasets.coco]: Conversion finished, #images: 234, #annotations: 2780 [07/12 01:31:58 d2.data.datasets.coco]: Caching COCO format annotations at 'Shared/cfu/models/c20495e1-6249-468f-a90a-985e980ac774/inference/dataset_test_coco_format.json' ... [07/12 01:31:59 d2.evaluation.evaluator]: Start inference on 234 batches [07/12 01:32:01 d2.evaluation.evaluator]: Inference done 11/234. Dataloading: 0.0035 s/iter. Inference: 0.1297 s/iter. Eval: 0.0531 s/iter. Total: 0.1863 s/iter. ETA=0:00:41 [07/12 01:32:06 d2.evaluation.evaluator]: Inference done 34/234. Dataloading: 0.0051 s/iter. Inference: 0.1273 s/iter. Eval: 0.0807 s/iter. Total: 0.2131 s/iter. ETA=0:00:42 [07/12 01:32:11 d2.evaluation.evaluator]: Inference done 65/234. Dataloading: 0.0047 s/iter. Inference: 0.1202 s/iter. Eval: 0.0617 s/iter. Total: 0.1866 s/iter. ETA=0:00:31 [07/12 01:32:17 d2.evaluation.evaluator]: Inference done 102/234. Dataloading: 0.0045 s/iter. Inference: 0.1142 s/iter. Eval: 0.0496 s/iter. Total: 0.1684 s/iter. ETA=0:00:22 [07/12 01:32:22 d2.evaluation.evaluator]: Inference done 137/234. Dataloading: 0.0044 s/iter. Inference: 0.1145 s/iter. Eval: 0.0437 s/iter. Total: 0.1627 s/iter. ETA=0:00:15 [07/12 01:32:27 d2.evaluation.evaluator]: Inference done 172/234. Dataloading: 0.0042 s/iter. Inference: 0.1158 s/iter. Eval: 0.0388 s/iter. Total: 0.1589 s/iter. ETA=0:00:09 [07/12 01:32:32 d2.evaluation.evaluator]: Inference done 225/234. Dataloading: 0.0037 s/iter. Inference: 0.1080 s/iter. Eval: 0.0318 s/iter. Total: 0.1437 s/iter. ETA=0:00:01 [07/12 01:32:33 d2.evaluation.evaluator]: Total inference time: 0:00:32.351187 (0.141272 s / iter per device, on 1 devices) [07/12 01:32:33 d2.evaluation.evaluator]: Total inference pure compute time: 0:00:24 (0.106709 s / iter per device, on 1 devices) [07/12 01:32:33 d2.evaluation.coco_evaluation]: Preparing results for COCO format ... [07/12 01:32:33 d2.evaluation.coco_evaluation]: Saving results to Shared/cfu/models/c20495e1-6249-468f-a90a-985e980ac774/inference/coco_instances_results.json [07/12 01:32:33 d2.evaluation.coco_evaluation]: Evaluating predictions with unofficial COCO API... Loading and preparing results... DONE (t=0.00s) creating index... index created! [07/12 01:32:33 d2.evaluation.fast_eval_api]: Evaluate annotation type bbox [07/12 01:32:33 d2.evaluation.fast_eval_api]: COCOeval_opt.evaluate() finished in 0.06 seconds. [07/12 01:32:33 d2.evaluation.fast_eval_api]: Accumulating evaluation results... [07/12 01:32:33 d2.evaluation.fast_eval_api]: COCOeval_opt.accumulate() finished in 0.01 seconds. Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.001 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.009 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.008 [07/12 01:32:33 d2.evaluation.coco_evaluation]: Evaluation results for bbox: | AP | AP50 | AP75 | APs | APm | APl | |:-----:|:------:|:------:|:-----:|:-----:|:-----:| | 0.017 | 0.073 | 0.002 | 0.016 | 0.030 | 0.227 | Loading and preparing results... DONE (t=0.02s) creating index... index created! [07/12 01:32:33 d2.evaluation.fast_eval_api]: Evaluate annotation type segm [07/12 01:32:33 d2.evaluation.fast_eval_api]: COCOeval_opt.evaluate() finished in 0.13 seconds. [07/12 01:32:33 d2.evaluation.fast_eval_api]: Accumulating evaluation results... [07/12 01:32:33 d2.evaluation.fast_eval_api]: COCOeval_opt.accumulate() finished in 0.01 seconds. Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.001 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.008 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.008 [07/12 01:32:33 d2.evaluation.coco_evaluation]: Evaluation results for segm: | AP | AP50 | AP75 | APs | APm | APl | |:-----:|:------:|:------:|:-----:|:-----:|:-----:| | 0.015 | 0.071 | 0.002 | 0.016 | 0.025 | 0.233 | [07/12 01:32:33 d2.engine.defaults]: Evaluation results for dataset_test in csv format: [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: main [07/12 01:32:33 d2.evaluation.testing]: copypaste: MAE (counts) 0,MAPE (% counts) 11+ [07/12 01:32:33 d2.evaluation.testing]: copypaste: 0.0000,4.8696 [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: mae [07/12 01:32:33 d2.evaluation.testing]: copypaste: 11+,51+,101+,0+,1+,0 [07/12 01:32:33 d2.evaluation.testing]: copypaste: 1.5263,1.6250,4.0000,0.4872,0.6552,0.0000 [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: mape [07/12 01:32:33 d2.evaluation.testing]: copypaste: 11+,51+,101+,1+,0 [07/12 01:32:33 d2.evaluation.testing]: copypaste: 4.8696,2.1965,3.5819,6.6990,nan [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: ratio [07/12 01:32:33 d2.evaluation.testing]: copypaste: 11+,51+,101+,0+,1+,0 [07/12 01:32:33 d2.evaluation.testing]: copypaste: 24.3590,6.8376,0.8547,100.0000,74.3590,25.6410 [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: bbox [07/12 01:32:33 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [07/12 01:32:33 d2.evaluation.testing]: copypaste: 0.0169,0.0728,0.0024,0.0163,0.0304,0.2268 [07/12 01:32:33 d2.evaluation.testing]: copypaste: Task: segm [07/12 01:32:33 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [07/12 01:32:33 d2.evaluation.testing]: copypaste: 0.0151,0.0710,0.0024,0.0161,0.0250,0.2332 [07/12 01:32:33 d2.utils.events]: eta: 1:57:34 iter: 55999 total_loss: 0.1169 loss_cls: 0.02125 loss_box_reg: 0.008282 loss_mask: 0.08076 loss_rpn_cls: 0.0007155 loss_rpn_loc: 0.001906 time: 0.4460 data_time: 0.0075 lr: 0.01 max_mem: 11345M [07/12 01:32:42 d2.utils.events]: eta: 1:57:25 iter: 56019 total_loss: 0.1438 loss_cls: 0.02627 loss_box_reg: 0.01023 loss_mask: 0.09723 loss_rpn_cls: 0.0008739 loss_rpn_loc: 0.004887 time: 0.4460 data_time: 0.0077 lr: 0.01 max_mem: 11345M [07/12 01:32:52 d2.utils.events]: eta: 1:57:22 iter: 56039 total_loss: 0.1892 loss_cls: 0.05523 loss_box_reg: 0.01809 loss_mask: 0.1071 loss_rpn_cls: 0.001002 loss_rpn_loc: 0.009079 time: 0.4460 data_time: 0.0218 lr: 0.01 max_mem: 11345M [07/12 01:33:01 d2.utils.events]: eta: 1:57:05 iter: 56059 total_loss: 0.2366 loss_cls: 0.06919 loss_box_reg: 0.01766 loss_mask: 0.1326 loss_rpn_cls: 0.00175 loss_rpn_loc: 0.01513 time: 0.4460 data_time: 0.0091 lr: 0.01 max_mem: 11345M [07/12 01:33:11 d2.utils.events]: eta: 1:57:04 iter: 56079 total_loss: 0.2322 loss_cls: 0.06314 loss_box_reg: 0.01654 loss_mask: 0.1259 loss_rpn_cls: 0.0009802 loss_rpn_loc: 0.01876 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:33:20 d2.utils.events]: eta: 1:56:55 iter: 56099 total_loss: 0.2308 loss_cls: 0.06212 loss_box_reg: 0.01369 loss_mask: 0.1242 loss_rpn_cls: 0.0008664 loss_rpn_loc: 0.01518 time: 0.4460 data_time: 0.0087 lr: 0.01 max_mem: 11345M [07/12 01:33:28 d2.utils.events]: eta: 1:56:46 iter: 56119 total_loss: 0.2202 loss_cls: 0.05569 loss_box_reg: 0.01555 loss_mask: 0.1299 loss_rpn_cls: 0.0002159 loss_rpn_loc: 0.00653 time: 0.4460 data_time: 0.0074 lr: 0.01 max_mem: 11345M [07/12 01:33:37 d2.utils.events]: eta: 1:56:34 iter: 56139 total_loss: 0.1764 loss_cls: 0.03215 loss_box_reg: 0.01407 loss_mask: 0.1191 loss_rpn_cls: 0.0005995 loss_rpn_loc: 0.005629 time: 0.4460 data_time: 0.0079 lr: 0.01 max_mem: 11345M [07/12 01:33:46 d2.utils.events]: eta: 1:56:42 iter: 56159 total_loss: 0.1778 loss_cls: 0.03448 loss_box_reg: 0.01189 loss_mask: 0.1052 loss_rpn_cls: 0.0004555 loss_rpn_loc: 0.004947 time: 0.4460 data_time: 0.0087 lr: 0.01 max_mem: 11345M [07/12 01:33:55 d2.utils.events]: eta: 1:56:34 iter: 56179 total_loss: 0.1679 loss_cls: 0.03963 loss_box_reg: 0.01348 loss_mask: 0.1169 loss_rpn_cls: 0.001884 loss_rpn_loc: 0.009099 time: 0.4460 data_time: 0.0078 lr: 0.01 max_mem: 11345M [07/12 01:34:04 d2.utils.events]: eta: 1:56:23 iter: 56199 total_loss: 0.2262 loss_cls: 0.06002 loss_box_reg: 0.01456 loss_mask: 0.1326 loss_rpn_cls: 0.00176 loss_rpn_loc: 0.0157 time: 0.4460 data_time: 0.0083 lr: 0.01 max_mem: 11345M [07/12 01:34:13 d2.utils.events]: eta: 1:56:14 iter: 56219 total_loss: 0.2058 loss_cls: 0.05331 loss_box_reg: 0.01728 loss_mask: 0.1241 loss_rpn_cls: 0.0008553 loss_rpn_loc: 0.0199 time: 0.4460 data_time: 0.0090 lr: 0.01 max_mem: 11345M [07/12 01:34:23 d2.utils.events]: eta: 1:56:09 iter: 56239 total_loss: 0.2486 loss_cls: 0.0609 loss_box_reg: 0.01941 loss_mask: 0.1307 loss_rpn_cls: 0.001822 loss_rpn_loc: 0.01551 time: 0.4460 data_time: 0.0074 lr: 0.01 max_mem: 11345M [07/12 01:34:32 d2.utils.events]: eta: 1:56:10 iter: 56259 total_loss: 0.2412 loss_cls: 0.06293 loss_box_reg: 0.01933 loss_mask: 0.1249 loss_rpn_cls: 0.0007813 loss_rpn_loc: 0.009812 time: 0.4460 data_time: 0.0081 lr: 0.01 max_mem: 11345M [07/12 01:34:41 d2.utils.events]: eta: 1:55:53 iter: 56279 total_loss: 0.2173 loss_cls: 0.04217 loss_box_reg: 0.01525 loss_mask: 0.1229 loss_rpn_cls: 0.00235 loss_rpn_loc: 0.01429 time: 0.4460 data_time: 0.0090 lr: 0.01 max_mem: 11345M [07/12 01:34:49 d2.utils.events]: eta: 1:55:42 iter: 56299 total_loss: 0.2156 loss_cls: 0.04108 loss_box_reg: 0.01467 loss_mask: 0.1237 loss_rpn_cls: 0.001051 loss_rpn_loc: 0.01096 time: 0.4460 data_time: 0.0075 lr: 0.01 max_mem: 11345M [07/12 01:34:59 d2.utils.events]: eta: 1:55:40 iter: 56319 total_loss: 0.257 loss_cls: 0.07825 loss_box_reg: 0.01948 loss_mask: 0.1326 loss_rpn_cls: 0.003547 loss_rpn_loc: 0.01474 time: 0.4460 data_time: 0.0090 lr: 0.01 max_mem: 11345M [07/12 01:35:07 d2.utils.events]: eta: 1:55:21 iter: 56339 total_loss: 0.2018 loss_cls: 0.04359 loss_box_reg: 0.01425 loss_mask: 0.1073 loss_rpn_cls: 0.001117 loss_rpn_loc: 0.008918 time: 0.4460 data_time: 0.0090 lr: 0.01 max_mem: 11345M [07/12 01:35:16 d2.utils.events]: eta: 1:55:12 iter: 56359 total_loss: 0.1739 loss_cls: 0.03893 loss_box_reg: 0.01233 loss_mask: 0.1126 loss_rpn_cls: 0.0004813 loss_rpn_loc: 0.003779 time: 0.4460 data_time: 0.0072 lr: 0.01 max_mem: 11345M [07/12 01:35:25 d2.utils.events]: eta: 1:55:09 iter: 56379 total_loss: 0.2093 loss_cls: 0.06248 loss_box_reg: 0.01734 loss_mask: 0.1192 loss_rpn_cls: 0.001699 loss_rpn_loc: 0.01419 time: 0.4460 data_time: 0.0088 lr: 0.01 max_mem: 11345M [07/12 01:35:34 d2.utils.events]: eta: 1:55:08 iter: 56399 total_loss: 0.1778 loss_cls: 0.04614 loss_box_reg: 0.0134 loss_mask: 0.1097 loss_rpn_cls: 0.0009438 loss_rpn_loc: 0.01291 time: 0.4460 data_time: 0.0083 lr: 0.01 max_mem: 11345M [07/12 01:35:43 d2.utils.events]: eta: 1:55:03 iter: 56419 total_loss: 0.1671 loss_cls: 0.04791 loss_box_reg: 0.01153 loss_mask: 0.1033 loss_rpn_cls: 0.0006425 loss_rpn_loc: 0.00368 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:35:52 d2.utils.events]: eta: 1:54:45 iter: 56439 total_loss: 0.1764 loss_cls: 0.0421 loss_box_reg: 0.01245 loss_mask: 0.107 loss_rpn_cls: 0.0006063 loss_rpn_loc: 0.006493 time: 0.4460 data_time: 0.0089 lr: 0.01 max_mem: 11345M [07/12 01:36:01 d2.utils.events]: eta: 1:54:42 iter: 56459 total_loss: 0.1802 loss_cls: 0.04233 loss_box_reg: 0.01541 loss_mask: 0.1087 loss_rpn_cls: 0.0005481 loss_rpn_loc: 0.00798 time: 0.4460 data_time: 0.0069 lr: 0.01 max_mem: 11345M [07/12 01:36:10 d2.utils.events]: eta: 1:54:30 iter: 56479 total_loss: 0.1868 loss_cls: 0.04195 loss_box_reg: 0.01782 loss_mask: 0.1082 loss_rpn_cls: 0.001694 loss_rpn_loc: 0.01846 time: 0.4460 data_time: 0.0088 lr: 0.01 max_mem: 11345M [07/12 01:36:18 d2.utils.events]: eta: 1:54:24 iter: 56499 total_loss: 0.2178 loss_cls: 0.05501 loss_box_reg: 0.01563 loss_mask: 0.127 loss_rpn_cls: 0.0008795 loss_rpn_loc: 0.01398 time: 0.4460 data_time: 0.0089 lr: 0.01 max_mem: 11345M [07/12 01:36:27 d2.utils.events]: eta: 1:53:50 iter: 56519 total_loss: 0.1668 loss_cls: 0.03967 loss_box_reg: 0.01277 loss_mask: 0.09428 loss_rpn_cls: 0.0009289 loss_rpn_loc: 0.00562 time: 0.4460 data_time: 0.0072 lr: 0.01 max_mem: 11345M [07/12 01:36:37 d2.utils.events]: eta: 1:53:54 iter: 56539 total_loss: 0.2153 loss_cls: 0.0553 loss_box_reg: 0.01673 loss_mask: 0.1219 loss_rpn_cls: 0.0007145 loss_rpn_loc: 0.009533 time: 0.4460 data_time: 0.0093 lr: 0.01 max_mem: 11345M [07/12 01:36:46 d2.utils.events]: eta: 1:53:54 iter: 56559 total_loss: 0.2056 loss_cls: 0.05546 loss_box_reg: 0.01681 loss_mask: 0.1252 loss_rpn_cls: 0.001095 loss_rpn_loc: 0.01251 time: 0.4460 data_time: 0.0088 lr: 0.01 max_mem: 11345M [07/12 01:36:55 d2.utils.events]: eta: 1:53:37 iter: 56579 total_loss: 0.1711 loss_cls: 0.04191 loss_box_reg: 0.01343 loss_mask: 0.1044 loss_rpn_cls: 0.001209 loss_rpn_loc: 0.01152 time: 0.4460 data_time: 0.0073 lr: 0.01 max_mem: 11345M [07/12 01:37:03 d2.utils.events]: eta: 1:53:29 iter: 56599 total_loss: 0.2211 loss_cls: 0.05814 loss_box_reg: 0.01569 loss_mask: 0.129 loss_rpn_cls: 0.001717 loss_rpn_loc: 0.01976 time: 0.4460 data_time: 0.0088 lr: 0.01 max_mem: 11345M [07/12 01:37:13 d2.utils.events]: eta: 1:53:24 iter: 56619 total_loss: 0.2186 loss_cls: 0.05173 loss_box_reg: 0.01801 loss_mask: 0.1142 loss_rpn_cls: 0.001285 loss_rpn_loc: 0.0193 time: 0.4460 data_time: 0.0097 lr: 0.01 max_mem: 11345M [07/12 01:37:21 d2.utils.events]: eta: 1:53:11 iter: 56639 total_loss: 0.1613 loss_cls: 0.03338 loss_box_reg: 0.01316 loss_mask: 0.1112 loss_rpn_cls: 0.0003977 loss_rpn_loc: 0.004656 time: 0.4460 data_time: 0.0072 lr: 0.01 max_mem: 11345M [07/12 01:37:30 d2.utils.events]: eta: 1:52:44 iter: 56659 total_loss: 0.1672 loss_cls: 0.02986 loss_box_reg: 0.01245 loss_mask: 0.1029 loss_rpn_cls: 0.0006512 loss_rpn_loc: 0.008434 time: 0.4460 data_time: 0.0075 lr: 0.01 max_mem: 11345M [07/12 01:37:39 d2.utils.events]: eta: 1:52:40 iter: 56679 total_loss: 0.223 loss_cls: 0.05242 loss_box_reg: 0.01731 loss_mask: 0.1349 loss_rpn_cls: 0.001059 loss_rpn_loc: 0.01187 time: 0.4460 data_time: 0.0085 lr: 0.01 max_mem: 11345M [07/12 01:37:48 d2.utils.events]: eta: 1:52:38 iter: 56699 total_loss: 0.1547 loss_cls: 0.02856 loss_box_reg: 0.0107 loss_mask: 0.1131 loss_rpn_cls: 0.001192 loss_rpn_loc: 0.006186 time: 0.4460 data_time: 0.0075 lr: 0.01 max_mem: 11345M [07/12 01:37:56 d2.utils.events]: eta: 1:52:24 iter: 56719 total_loss: 0.1958 loss_cls: 0.04971 loss_box_reg: 0.01445 loss_mask: 0.1476 loss_rpn_cls: 0.001104 loss_rpn_loc: 0.01117 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:38:05 d2.utils.events]: eta: 1:52:15 iter: 56739 total_loss: 0.1769 loss_cls: 0.03637 loss_box_reg: 0.01332 loss_mask: 0.1161 loss_rpn_cls: 0.001293 loss_rpn_loc: 0.00866 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:38:15 d2.utils.events]: eta: 1:52:25 iter: 56759 total_loss: 0.227 loss_cls: 0.05001 loss_box_reg: 0.01671 loss_mask: 0.1225 loss_rpn_cls: 0.001918 loss_rpn_loc: 0.01686 time: 0.4460 data_time: 0.0091 lr: 0.01 max_mem: 11345M [07/12 01:38:24 d2.utils.events]: eta: 1:52:20 iter: 56779 total_loss: 0.2195 loss_cls: 0.04918 loss_box_reg: 0.01663 loss_mask: 0.1289 loss_rpn_cls: 0.002565 loss_rpn_loc: 0.01142 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:38:33 d2.utils.events]: eta: 1:52:05 iter: 56799 total_loss: 0.2651 loss_cls: 0.08386 loss_box_reg: 0.01853 loss_mask: 0.143 loss_rpn_cls: 0.004439 loss_rpn_loc: 0.0167 time: 0.4460 data_time: 0.0089 lr: 0.01 max_mem: 11345M [07/12 01:38:42 d2.utils.events]: eta: 1:51:59 iter: 56819 total_loss: 0.2422 loss_cls: 0.05831 loss_box_reg: 0.01871 loss_mask: 0.1438 loss_rpn_cls: 0.002861 loss_rpn_loc: 0.007384 time: 0.4460 data_time: 0.0077 lr: 0.01 max_mem: 11345M [07/12 01:38:51 d2.utils.events]: eta: 1:51:50 iter: 56839 total_loss: 0.2292 loss_cls: 0.0434 loss_box_reg: 0.01323 loss_mask: 0.1351 loss_rpn_cls: 0.0023 loss_rpn_loc: 0.01812 time: 0.4460 data_time: 0.0088 lr: 0.01 max_mem: 11345M [07/12 01:38:59 d2.utils.events]: eta: 1:51:50 iter: 56859 total_loss: 0.242 loss_cls: 0.0578 loss_box_reg: 0.01444 loss_mask: 0.1273 loss_rpn_cls: 0.001829 loss_rpn_loc: 0.008506 time: 0.4460 data_time: 0.0078 lr: 0.01 max_mem: 11345M [07/12 01:39:08 d2.utils.events]: eta: 1:51:33 iter: 56879 total_loss: 0.2622 loss_cls: 0.06702 loss_box_reg: 0.01816 loss_mask: 0.1327 loss_rpn_cls: 0.0005544 loss_rpn_loc: 0.01144 time: 0.4460 data_time: 0.0081 lr: 0.01 max_mem: 11345M [07/12 01:39:17 d2.utils.events]: eta: 1:51:27 iter: 56899 total_loss: 0.2804 loss_cls: 0.0782 loss_box_reg: 0.02008 loss_mask: 0.1333 loss_rpn_cls: 0.002271 loss_rpn_loc: 0.01798 time: 0.4460 data_time: 0.0095 lr: 0.01 max_mem: 11345M [07/12 01:39:26 d2.utils.events]: eta: 1:51:11 iter: 56919 total_loss: 0.2591 loss_cls: 0.05679 loss_box_reg: 0.01635 loss_mask: 0.1276 loss_rpn_cls: 0.0007462 loss_rpn_loc: 0.009331 time: 0.4460 data_time: 0.0074 lr: 0.01 max_mem: 11345M [07/12 01:39:35 d2.utils.events]: eta: 1:51:12 iter: 56939 total_loss: 0.2809 loss_cls: 0.06402 loss_box_reg: 0.02063 loss_mask: 0.1455 loss_rpn_cls: 0.002003 loss_rpn_loc: 0.02014 time: 0.4460 data_time: 0.0084 lr: 0.01 max_mem: 11345M [07/12 01:39:44 d2.utils.events]: eta: 1:51:03 iter: 56959 total_loss: 0.2491 loss_cls: 0.06104 loss_box_reg: 0.01602 loss_mask: 0.1299 loss_rpn_cls: 0.001515 loss_rpn_loc: 0.0185 time: 0.4460 data_time: 0.0087 lr: 0.01 max_mem: 11345M [07/12 01:39:52 d2.utils.events]: eta: 1:50:48 iter: 56979 total_loss: 0.2124 loss_cls: 0.04628 loss_box_reg: 0.01474 loss_mask: 0.1145 loss_rpn_cls: 0.0009609 loss_rpn_loc: 0.006586 time: 0.4460 data_time: 0.0081 lr: 0.01 max_mem: 11345M [07/12 01:40:02 d2.utils.events]: eta: 1:50:46 iter: 56999 total_loss: 0.2877 loss_cls: 0.07403 loss_box_reg: 0.01935 loss_mask: 0.1316 loss_rpn_cls: 0.001429 loss_rpn_loc: 0.01994 time: 0.4460 data_time: 0.0089 lr: 0.01 max_mem: 11345M [07/12 01:40:10 d2.utils.events]: eta: 1:50:46 iter: 57019 total_loss: 0.1763 loss_cls: 0.03502 loss_box_reg: 0.01379 loss_mask: 0.1164 loss_rpn_cls: 0.000419 loss_rpn_loc: 0.006528 time: 0.4460 data_time: 0.0073 lr: 0.01 max_mem: 11345M [07/12 01:40:20 d2.utils.events]: eta: 1:50:33 iter: 57039 total_loss: 0.2803 loss_cls: 0.08492 loss_box_reg: 0.02088 loss_mask: 0.1424 loss_rpn_cls: 0.003215 loss_rpn_loc: 0.01601 time: 0.4460 data_time: 0.0092 lr: 0.01 max_mem: 11345M [07/12 01:40:28 d2.utils.events]: eta: 1:50:16 iter: 57059 total_loss: 0.1912 loss_cls: 0.04257 loss_box_reg: 0.01312 loss_mask: 0.1217 loss_rpn_cls: 0.001922 loss_rpn_loc: 0.00909 time: 0.4460 data_time: 0.0079 lr: 0.01 max_mem: 11345M [07/12 01:40:38 d2.utils.events]: eta: 1:50:10 iter: 57079 total_loss: 0.2533 loss_cls: 0.07634 loss_box_reg: 0.01825 loss_mask: 0.1199 loss_rpn_cls: 0.002404 loss_rpn_loc: 0.01514 time: 0.4460 data_time: 0.0090 lr: 0.01 max_mem: 11345M [07/12 01:40:46 d2.utils.events]: eta: 1:49:50 iter: 57099 total_loss: 0.4486 loss_cls: 0.08131 loss_box_reg: 0.0095 loss_mask: 0.2016 loss_rpn_cls: 0.03777 loss_rpn_loc: 0.01597 time: 0.4460 data_time: 0.0077 lr: 0.01 max_mem: 11345M ERROR [07/12 01:40:51 d2.engine.train_loop]: Exception during training: Traceback (most recent call last): File "/databricks/python/lib/python3.8/site-packages/detectron2/engine/train_loop.py", line 149, in train self.run_step() File "/databricks/python/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 494, in run_step self._trainer.run_step() File "/databricks/python/lib/python3.8/site-packages/detectron2/engine/train_loop.py", line 273, in run_step loss_dict = self.model(data) File "/databricks/python/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/databricks/python/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 163, in forward _, detector_losses = self.roi_heads(images, features, proposals, gt_instances) File "/databricks/python/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/databricks/python/lib/python3.8/site-packages/detectron2/modeling/roi_heads/roi_heads.py", line 739, in forward losses = self._forward_box(features, proposals) File "/databricks/python/lib/python3.8/site-packages/detectron2/modeling/roi_heads/roi_heads.py", line 804, in _forward_box losses = self.box_predictor.losses(predictions, proposals) File "/databricks/python/lib/python3.8/site-packages/detectron2/modeling/roi_heads/fast_rcnn.py", line 323, in losses "loss_box_reg": self.box_reg_loss( File "/databricks/python/lib/python3.8/site-packages/detectron2/modeling/roi_heads/fast_rcnn.py", line 358, in box_reg_loss loss_box_reg = giou_loss(fg_pred_boxes, gt_boxes[fg_inds], reduction="sum") File "/databricks/python/lib/python3.8/site-packages/fvcore/nn/giou_loss.py", line 32, in giou_loss assert (x2 >= x1).all(), "bad box: x1 larger than x2" AssertionError: bad box: x1 larger than x2 [07/12 01:40:51 d2.engine.hooks]: Overall training speed: 57109 iterations in 7:04:29 (0.4460 s / it) [07/12 01:40:51 d2.engine.hooks]: Total training time: 7:16:06 (0:11:36 on hooks) [07/12 01:40:51 d2.utils.events]: eta: 1:49:42 iter: 57111 total_loss: 0.8432 loss_cls: 0.07376 loss_box_reg: 0.00708 loss_mask: 0.2679 loss_rpn_cls: 0.3349 loss_rpn_loc: 0.04453 time: 0.4460 data_time: 0.0079 lr: 0.01 max_mem: 11345M

AssertionError: A prediction has class=56, but the dataset only has 4 classes, and the predicted class id should be in [0, 3].

Where in the detectron2 config will I change the class number?

Have anyones trained this code in pascal voc? I had implemented a version of tensorflow codes, but it performed slightly worsh compared with RetinaNet and FCOS. And there is my codes: https://github.com/JiXuKong/YOLOF

AssertionError: Attribute 'thing_dataset_id_to_contiguous_id' in the metadata of 'coco_2017_train' cannot be set to a different value! {1: 0} != {0: 0} 修改detectron2/data/datasets中builtin_meta.py文件，但是训练报错，是不是类别不匹配，configs/default.py文件中的类别数也改过来了。（训练一类数据集）