Google AI Open Images - Object Detection Track: Open Solution

minerva.ml

Last update: Jun 22, 2022

Related tags

Deep Learning challenge data-science machine-learning deep-neural-networks deep-learning pipeline reproducible-research pipeline-framework object-detection reproducibility reproducible-experiments mask-rcnn detection-network retina-net google-ai-challange

Overview

Google AI Open Images - Object Detection Track: Open Solution

This is an open solution to the Google AI Open Images - Object Detection Track 😃

More competitions 🎇

Check collection of public projects 🎁 , where you can find multiple Kaggle competitions with code, experiments and outputs.

Our goals

We are building entirely open solution to this competition. Specifically:

Learning from the process - updates about new ideas, code and experiments is the best way to learn data science. Our activity is especially useful for people who wants to enter the competition, but lack appropriate experience.
Encourage more Kagglers to start working on this competition.
Deliver open source solution with no strings attached. Code is available on our GitHub repository 💻 . This solution should establish solid benchmark, as well as provide good base for your custom ideas and experiments. We care about clean code 😃
We are opening our experiments as well: everybody can have live preview on our experiments, parameters, code, etc. Check: Google-AI-Object-Detection-Challenge 📈 and images below:

UNet training monitor 📊	Predicted bounding boxes 📊

Disclaimer

In this open source solution you will find references to the neptune.ml. It is free platform for community Users, which we use daily to keep track of our experiments. Please note that using neptune.ml is not necessary to proceed with this solution. You may run it as plain Python script 🐍 .

How to start?

Learn about our solutions

Check Kaggle forum and participate in the discussions.
Check our Wiki pages 🐬 , where we describe our work. Below are link to specific solutions:

link to code	link to description
solution-1	palm-tree 🌴

Dataset for this competition

This competition is special, because it used Open Images Dataset V4, which is quite large: >1.8M images and >0.5TB 😲 To make it more approachable, we are hosting entire dataset in the neptune's public directory 😎 . You can use this dataset in neptune.ml with no additional setup 👍 .

Start experimenting with ready-to-use code

You can jump start your participation in the competition by using our starter pack. Installation instruction below will guide you through the setup.

Installation

Fast Track

Clone repository, install requirements (check _requirements.txt)

pip3 install -r requirements.txt

Register to the neptune.ml (if you wish to use it) and create your project, for example Google-AI-Object-Detection-Challenge.
Train RetinaNet:

🐹

neptune send --worker m-4p100 \
--environment pytorch-0.3.1-gpu-py3 \
--config configs/neptune.yaml \
main.py train --pipeline_name retinanet

🔱

neptune run main.py train --pipeline_name retinanet

🐍

python main.py -- train --pipeline_name retinanet

Evaluate/Predict RetinaNet:

Note in case of memory trouble go to neptune.yaml and change batch_size_inference: 1

🐹 With cloud environment you need to change the experiment directory to the one that you have just trained. Let's assume that your experiment id was GAI-14. You should go to neptune.yaml and change:

  experiment_dir:  /output/experiment
  clone_experiment_dir_from:  /input/GAI-14/output/experiment

neptune send --worker m-4p100 \
--environment pytorch-0.3.1-gpu-py3 \
--config configs/neptune.yaml \
--input /GAI-14 \
main.py evaluate_predict --pipeline_name retinanet --chunk_size 100

🔱

neptune run main.py train --pipeline_name retinanet --chunk_size 100

🐍

python main.py -- train --pipeline_name retinanet --chunk_size 100

Get involved

You are welcome to contribute your code and ideas to this open solution. To get started:

Check competition project on GitHub to see what we are working on right now.
Express your interest in particular task by writing comment in this task, or by creating new one with your fresh idea.
We will get back to you quickly in order to start working together.
Check CONTRIBUTING for some more information.

User support

There are several ways to seek help:

Kaggle discussion is our primary way of communication.
Read project's Wiki, where we publish descriptions about the code, pipelines and supporting tools such as neptune.ml.
Submit an issue directly in this repo.

Comments

Evaluating loss function: TypeError: can only concatenate tuple (not "dict") to tuple

I am running the latest master branch (offline), and when the code gets to the training point it crashes when trying to forward() the model and evaluate the loss function:

neptune: Executing in Offline Mode.
2018-08-15 18-34-40 google-ai-odt >>> training
2018-08-15 18-35-03 google-ai-odt >>> Training on a reduced class subset: ['Person', 'Car', 'Dress', 'Footwear']
2018-08-15 18:35:05 steppy >>> initializing Step label_encoder...
2018-08-15 18:35:05 steppy >>> initializing Step label_encoder...
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> Step label_encoder initialized
2018-08-15 18:35:05 steppy >>> Step label_encoder initialized
2018-08-15 18:35:05 steppy >>> initializing Step loader...
2018-08-15 18:35:05 steppy >>> initializing Step loader...
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> Step loader initialized
2018-08-15 18:35:05 steppy >>> Step loader initialized
neptune: Executing in Offline Mode.
2018-08-15 18:35:07 steppy >>> initializing Step retinanet...
2018-08-15 18:35:07 steppy >>> initializing Step retinanet...
2018-08-15 18:35:07 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:07 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:07 steppy >>> done: initializing experiment directories
2018-08-15 18:35:07 steppy >>> done: initializing experiment directories
2018-08-15 18:35:07 steppy >>> Step retinanet initialized
2018-08-15 18:35:07 steppy >>> Step retinanet initialized
2018-08-15 18:35:07 steppy >>> cleaning cache...
2018-08-15 18:35:07 steppy >>> cleaning cache...
2018-08-15 18:35:07 steppy >>> cleaning cache done
2018-08-15 18:35:07 steppy >>> cleaning cache done
2018-08-15 18:35:07 steppy >>> Step label_encoder, adapting inputs...
2018-08-15 18:35:07 steppy >>> Step label_encoder, adapting inputs...
2018-08-15 18:35:07 steppy >>> Step label_encoder, fitting and transforming...
2018-08-15 18:35:07 steppy >>> Step label_encoder, fitting and transforming...
2018-08-15 18:35:10 steppy >>> Step label_encoder, persisting transformer to the experiments/transformers/label_encoder
2018-08-15 18:35:10 steppy >>> Step label_encoder, persisting transformer to the experiments/transformers/label_encoder
2018-08-15 18:35:10 steppy >>> Step loader, adapting inputs...
2018-08-15 18:35:10 steppy >>> Step loader, adapting inputs...
2018-08-15 18:35:10 steppy >>> Step loader, transforming...
2018-08-15 18:35:10 steppy >>> Step loader, transforming...
2018-08-15 18:35:10 steppy >>> Step retinanet, unpacking inputs...
2018-08-15 18:35:10 steppy >>> Step retinanet, unpacking inputs...
2018-08-15 18:35:10 steppy >>> Step retinanet, fitting and transforming...
2018-08-15 18:35:10 steppy >>> Step retinanet, fitting and transforming...
2018-08-15 18:35:13 steppy >>> starting training...
2018-08-15 18:35:13 steppy >>> starting training...
2018-08-15 18:35:13 steppy >>> initial lr: 1e-05
2018-08-15 18:35:13 steppy >>> initial lr: 1e-05
2018-08-15 18:35:13 steppy >>> epoch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 batch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 batch 0 ...
Traceback (most recent call last):
  File "main.py", line 78, in <module>
    main()
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "main.py", line 16, in train
    pipeline_manager.train(pipeline_name, dev_mode)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/pipeline_manager.py", line 21, in train
    train(pipeline_name, dev_mode)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/pipeline_manager.py", line 85, in train
    pipeline.fit_transform(data)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/steppy_dev/base.py", line 280, in fit_transform
    step_output_data = self._cached_fit_transform(step_inputs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/steppy_dev/base.py", line 390, in _cached_fit_transform
    step_output_data = self.transformer.fit_transform(**step_inputs)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/steppy/base.py", line 605, in fit_transform
    self.fit(*args, **kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/models.py", line 32, in fit
    metrics = self._fit_loop(data)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/models.py", line 63, in _fit_loop
    batch_loss = loss_function(outputs_batch, target) * weight
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 137, in forward
    outputs = _criterion_parallel_apply(replicas, inputs, targets, kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 192, in _criterion_parallel_apply
    raise output
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 167, in _worker
    output = module(*(input + target), **kwargs)
TypeError: can only concatenate tuple (not "dict") to tuple

It looks like the "target" variable used for the loss function is supposed to be a tuple, but instead it is a dictionary. I have to admit I'm not sure what exactly's causing this, but I wanted to see if you have any immediate ideas before I spend time going through the code line by line. Execution command is just: python main.py -- train --pipeline_name retinanet, and the whole config has been filled out with (supposedly) the correct files.

Thanks!

opened by mxbi 2

Bug in script 'src/object_detection/dataset_tools/oid_hierarchical_labels_expansion.py'
The above script (src/object_detection/dataset_tools/oid_hierarchical_labels_expansion.py) doesn't expand hierarchically image-level labels and boxes as it is supposed to be doing. It just replicates the input file and outputs a duplicated file.

I believe the bug is in the below part of the code :

with open(parsed_args.input_annotations, 'r') as source: with open(parsed_args.output_annotations, 'w') as target: header = None for line in source: if not header: target.writelines(line) continue if labels_file: expanded_lines = expansion_generator.expand_labels_from_csv(line) else: expanded_lines = expansion_generator.expand_boxes_from_csv(line) target.writelines(expanded_lines)

The header is always None and doesn't get set to any other value, hence script just replicates the input file. Possible solution is to just add header=line just before the continue statement.
opened by manojsukhavasi 2
Let's visualise the results!

As our project is prepared for the community, I think we should take care of some community-focused issues, like possibility to visualise the results. In my opinion a proper jupyter notebook or separate cli command for writing to .png would be sufficient, like neptune run main.py visualise_results.
feature-request

opened by apyskir 2
Dev vis custom class

Adding vis for one (or more classes)

neptune run --config "./configs/neptune_config_local.yaml" main.py -- visualize --pipeline_name retinanet --classes_to_visualize='Picture frame,Cat'

opened by i008 1
generating submission in dev mode

If we don't run predict on whole test set submission fails because every image from test set has to have corresponding row in submission file. Adding rows with image id and empty prediction string should be enough.
bug

opened by taraspiotr 1
data decoder doesn't work with 0 predicted boxes
ids = score > CLS_THRESH ids = ids.nonzero().squeeze() # [#obj,] keep = box_nms(boxes[ids], score[ids], threshold=NMS_THRESH) return boxes[ids][keep], labels[ids][keep]

When there are no positive ids (where score > CSL_TRESH) code tries to index with empty tensor and that raises a PyTorch error
bug
opened by taraspiotr 1
train on subset of classes

In COCO dataset there are 80 different classes, some architectures that worked previously on COCO might not work on Open Images in scale 1:1. Having an option to train on just a subset of classes (e.g. 80) and sampler from #1 that will generate images with instances from this subset could help evaluating base architectures.
feature-request

opened by taraspiotr 1
Grouping images into batches of similar aspect ratios

So this little hack seems quite useful and might give a (small) boost. I've only seen it beeing implemented here: https://github.com/fizyr/keras-retinanet/blob/ebb4512a0ab0efc08bdb8e800e77ab89b6ca2cc1/keras_retinanet/preprocessing/generator.py#L210

The idea is to avoid destructive resizing by precomputing aspect ratios (image dimensions) for all images, and then yielding images with similar AR's to use as little padding as possible.

opened by i008 1
Bump protobuf from 3.6.0 to 3.15.0
Bumps protobuf from 3.6.0 to 3.15.0.

Release notes

Sourced from protobuf's releases.

Protocol Buffers v3.15.0

Protocol Compiler

Optional fields for proto3 are enabled by default, and no longer require the --experimental_allow_proto3_optional flag.

C++

MessageDifferencer: fixed bug when using custom ignore with multiple unknown fields

Use init_seg in MSVC to push initialization to an earlier phase.

Runtime no longer triggers -Wsign-compare warnings.

Fixed -Wtautological-constant-out-of-range-compare warning.

DynamicCastToGenerated works for nullptr input for even if RTTI is disabled

Arena is refactored and optimized.

Clarified/specified that the exact value of Arena::SpaceAllocated() is an implementation detail users must not rely on. It should not be used in unit tests.

Change the signature of Any::PackFrom() to return false on error.

Add fast reflection getter API for strings.

Constant initialize the global message instances

Avoid potential for missed wakeup in UnknownFieldSet

Now Proto3 Oneof fields have "has" methods for checking their presence in C++.

Bugfix for NVCC

Return early in _InternalSerialize for empty maps.

Adding functionality for outputting map key values in proto path logging output (does not affect comparison logic) and stop printing 'value' in the path. The modified print functionality is in the MessageDifferencer::StreamReporter.

Fixed protocolbuffers/protobuf#8129

Ensure that null char symbol, package and file names do not result in a crash.

Constant initialize the global message instances

Pretty print 'max' instead of numeric values in reserved ranges.

Removed remaining instances of std::is_pod, which is deprecated in C++20.

Changes to reduce code size for unknown field handling by making uncommon cases out of line.

Fix std::is_pod deprecated in C++20 (#7180)

Fix some -Wunused-parameter warnings (#8053)

Fix detecting file as directory on zOS issue #8051 (#8052)

Don't include sys/param.h for _BYTE_ORDER (#8106)

remove CMAKE_THREAD_LIBS_INIT from pkgconfig CFLAGS (#8154)

Fix TextFormatMapTest.DynamicMessage issue#5136 (#8159)

Fix for compiler warning issue#8145 (#8160)

fix: support deprecated enums for GCC < 6 (#8164)

Fix some warning when compiling with Visual Studio 2019 on x64 target (#8125)

Python

Provided an override for the reverse() method that will reverse the internal collection directly instead of using the other methods of the BaseContainer.

MessageFactory.CreateProtoype can be overridden to customize class creation.

... (truncated)

Commits

ae50d9b Update protobuf version

8260126 Update protobuf version

c741c46 Resovled issue in the .pb.cc files

eef2764 Resolved an issue where NO_DESTROY and CONSTINIT were in incorrect order

0040102 Updated collect_all_artifacts.sh for Ubuntu Xenial

26cb6a7 Delete root-owned files in Kokoro builds

1e924ef Update port_def.inc

9a80cf1 Update coded_stream.h

a97c4f4 Merge pull request #8276 from haberman/php-warning

44cd75d Merge pull request #8282 from haberman/changelog

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump pillow from 5.2.0 to 6.2.0
Bumps pillow from 5.2.0 to 6.2.0.

Release notes

Sourced from pillow's releases.

6.2.0

https://pillow.readthedocs.io/en/stable/releasenotes/6.2.0.html

6.1.0

https://pillow.readthedocs.io/en/stable/releasenotes/6.1.0.html

6.0.0

No release notes provided.

5.4.1

No release notes provided.

5.4.0

No release notes provided.

5.3.0

No release notes provided.

Changelog

Sourced from pillow's changelog.

6.2.0 (2019-10-01)

Catch buffer overruns #4104 [radarhere]

Initialize rows_per_strip when RowsPerStrip tag is missing #4034 [cgohlke, radarhere]

Raise error if TIFF dimension is a string #4103 [radarhere]

Added decompression bomb checks #4102 [radarhere]

Fix ImageGrab.grab DPI scaling on Windows 10 version 1607+ #4000 [nulano, radarhere]

Corrected negative seeks #4101 [radarhere]

Added argument to capture all screens on Windows #3950 [nulano, radarhere]

Updated warning to specify when Image.frombuffer defaults will change #4086 [radarhere]

Changed WindowsViewer format to PNG #4080 [radarhere]

Use TIFF orientation #4063 [radarhere]

Raise the same error if a truncated image is loaded a second time #3965 [radarhere]

Lazily use ImageFileDirectory_v1 values from Exif #4031 [radarhere]

Improved HSV conversion #4004 [radarhere]

Added text stroking #3978 [radarhere, hugovk]

No more deprecated bdist_wininst .exe installers #4029 [hugovk]

Do not allow floodfill to extend into negative coordinates #4017 [radarhere]

... (truncated)

Commits

8a30d13 Updated CHANGES.rst [ci skip]

75602d1 6.2.0 version bump

4756af9 Updated CHANGES.rst [ci skip]

cc16025 Merge pull request #4104 from radarhere/overrun

fb84701 Merge pull request #4034 from cgohlke/patch-1

b9693a5 Merge pull request #4103 from radarhere/dimension

f228d0c Merge pull request #4102 from radarhere/decompression

aaf2c42 Merge pull request #4000 from nulano/dpi_fix

b36c1bc Merge pull request #4101 from radarhere/negative_seek

9a977b9 Raise error if dimension is a string

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Meaning of Sum

I got your training script to run, but I was wondering what the meaning of "sum" is. I.e. "2018-08-18 09:23:40 steps >>> epoch 1 batch 12 sum: 10.1"

Is that the sum of the losses in the given batch or how can I interpret it? I looked into the code and couldn't find the information. I'm assuming it's part of the steppy toolkit?

opened by downrealbad 1
Handling experiment directories
I belive this https://github.com/neptune-ml/open-solution-googleai-object-detection/blob/master/src/pipeline_manager.py#L36 line is VERY dangerous, it can easly delete someones home-dir if he does not understands what we ask for.

We could create new dir for each run (in a basedir provided)

We should store the config file in the experiment_dir too (just make a copy?)

bug feature-request
opened by i008 0

Releases(solution-1)

solution-1(Aug 9, 2018)

Description: https://github.com/neptune-ml/open-solution-googleai-object-detection/wiki/RetinaNet-with-sampler
Source code(tar.gz)
Source code(zip)