Kaggle | 9th place single model solution for TGS Salt Identification Challenge

Overview

UNet for segmenting salt deposits from seismic images with PyTorch.

General

We, tugstugi and xuyuan, have participated in the Kaggle competition TGS Salt Identification Challenge and reached the 9-th place. This repository contains a simplified and cleaned up version of our team's code partially based on the ideas of Heng Cherkeng's discussion on the Kaggle discussion board.

We have used a single UNet model with a SENet154 encoder which has a single fold score of 0.882. With 10 folds using reflective padding and another 10 folds with resizing, we got 0.890. The final private LB score 0.892 was achieved by post processing on the model's output.

Features

def symmetric_lovasz(outputs, targets):
    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2

Training

  1. Download and extract the dataset
    • copy train.csv into datasets/
    • copy train images and masks into datasets/train/
    • copy test images into datasets/test/
  2. Train SENet154-Unet for 250 epochs on 2x P100: python train.py --vtf --pretrained imagenet --loss-on-center --batch-size 32 --optim adamw --learning-rate 5e-4 --lr-scheduler noam --basenet senet154 --max-epochs 250 --data-fold fold0 --log-dir runs/fold0 --resume runs/fold0/checkpoints/last-checkpoint-fold0.pth
    • tensorboard logs, checkpoints and models are saved under runs/
    • start tensorboard with tensorboard --logdir runs
    • training log of a LB0.883 model is provided under runs/lb0.883_fold0/
  3. Do SWA on the best loss, accuracy and kaggle metrics models: python swa.py --input runs/fold0/models --output fold0_swa.pth
  4. Create a Kaggle submission: python test.py --tta fold0_swa.pth --output-prefix fold0
    • a submission file fold0-submission.csv should be created now
You might also like...
Code for 1st place solution in Sleep AI Challenge SNU Hospital
Code for 1st place solution in Sleep AI Challenge SNU Hospital

Sleep AI Challenge SNU Hospital 2021 Code for 1st place solution for Sleep AI Challenge (Note that the code is not fully organized) Refer to the notio

 Waymo motion prediction challenge 2021: 3rd place solution
Waymo motion prediction challenge 2021: 3rd place solution

Waymo motion prediction challenge 2021: 3rd place solution 📜 Technical report 🗨️ Presentation 🎉 Announcement 🛆Motion Prediction Channel Website 🛆

4th place solution to datafactory challenge by Intermarché.

Solution to Datafactory challenge by Intermarché. 4th place solution to datafactory challenge by Intermarché. The objective of the challenge is to pre

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime
1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

4th place solution for the SIGIR 2021 challenge.

SIGIR-2021 (Tinkoff.AI) How to start Download train and test data: https://sigir-ecom.github.io/data-task.html Place it under sigir-2021/data/. Run py

 Meli Data Challenge 2021 - First Place Solution
Meli Data Challenge 2021 - First Place Solution

My solution for the Meli Data Challenge 2021

The sixth place winning solution (6/220) in 2021 Gaofen Challenge.
The sixth place winning solution (6/220) in 2021 Gaofen Challenge.

SwinTransformer + OBBDet The sixth place winning solution (6/220) in the track of Fine-grained Object Recognition in High-Resolution Optical Images, 2

Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL BASALT Challenge.

KAIROS MineRL BASALT Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL B

Kohei's 5th place solution for xview3 challenge

xview3-kohei-solution Usage This repository assumes that the given data set is stored in the following locations: $ ls data/input/xview3/*.csv data/in

Comments
  • About bn_update

    About bn_update

    Hi, @tugstugi I know we should run one additional pass over the training data to get the running mean and running var. But why change momentum?

            momentum = b / (n + b)
            for module in momenta.keys():
                module.momentum = momentum
    
    opened by zllrunning 2
  • Cannot allocate memory

    Cannot allocate memory

    I use python train.py --vtf --pretrained imagenet --loss-on-center --batch-size 6 --optim adamw --learning-rate 5e-4 --lr-scheduler noam --basenet resnet34 --max-epochs 100 --data-fold fold0 --log-dir runs/fold0 --resume runs/fold0/checkpoints/last-checkpoint-fold0.pth to train my dataset. My train data have 71712 images and valid data have 7968 images. I got this error: 0f69702d7a503b0aeb6e055814831c9

    opened by zxshi 1
  • About `NoamLR`

    About `NoamLR`

    https://github.com/tugstugi/pytorch-saltnet/blob/a3e63b357f975924e75b5db24ee528c5797f5efb/utils/lr_scheduler.py#L36-L39

    The custom NoamLR results in same LR in the first two steps, like:

    warmup_steps = 10
    lr = 0.01
    
    • step -- before (current step LR) -- after (next step LR)
    • 0 -- 0.001 -- 0.001
    • 1 -- 0.001 -- 0.002
    • 2 -- 0.002 -- 0.003

    There are two ways to fix this:

    • last_epoch = self.last_epoch + 1, like ESPnet
      def get_lr(self): 
          last_epoch = self.last_epoch + 1
          scale = self.warmup_steps ** 0.5 * min(last_epoch ** (-0.5), last_epoch * self.warmup_steps ** (-1.5)) 
          return [base_lr * scale for base_lr in self.base_lrs] 
      
    • use LambdaLR directly
      noam_scale = lambda epoch: (warmup_steps ** 0.5) * min((epoch + 1) ** -0.5, (epoch + 1) * (warmup_steps ** -1.5))
      scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=noam_scale)
      

    Of course, the above two approaches are equivalent.

    opened by ShaneTian 0
Owner
Erdene-Ochir Tuguldur
Берлиний Техникийн Их Сургууль
Erdene-Ochir Tuguldur
9th place solution in "Santa 2020 - The Candy Cane Contest"

Santa 2020 - The Candy Cane Contest My solution in this Kaggle competition "Santa 2020 - The Candy Cane Contest", 9th place. Basic Strategy In this co

toshi_k 22 Nov 26, 2021
9th place solution

AllDataAreExt-Galixir-Kaggle-HPA-2021-Solution Team Members Qishen Ha is Master of Engineering from the University of Tokyo. Machine Learning Engineer

daishu 5 Nov 18, 2021
10th place solution for Google Smartphone Decimeter Challenge at kaggle.

Under refactoring 10th place solution for Google Smartphone Decimeter Challenge at kaggle. Google Smartphone Decimeter Challenge Global Navigation Sat

null 12 Oct 25, 2022
The 1st place solution of track2 (Vehicle Re-Identification) in the NVIDIA AI City Challenge at CVPR 2021 Workshop.

AICITY2021_Track2_DMT The 1st place solution of track2 (Vehicle Re-Identification) in the NVIDIA AI City Challenge at CVPR 2021 Workshop. Introduction

Hao Luo 91 Dec 21, 2022
7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

kaggle-hpa-2021-7th-place-solution Code for 7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle. A description of the met

null 8 Jul 9, 2021
Xview3 solution - XView3 challenge, 2nd place solution

Xview3, 2nd place solution https://iuu.xview.us/ test split aggregate score publ

Selim Seferbekov 24 Nov 23, 2022
Kaggle Lyft Motion Prediction for Autonomous Vehicles 4th place solution

Lyft Motion Prediction for Autonomous Vehicles Code for the 4th place solution of Lyft Motion Prediction for Autonomous Vehicles on Kaggle. Discussion

null 44 Jun 27, 2022
My 1st place solution at Kaggle Hotel-ID 2021

1st place solution at Kaggle Hotel-ID My 1st place solution at Kaggle Hotel-ID to Combat Human Trafficking 2021. https://www.kaggle.com/c/hotel-id-202

Kohei Ozaki 18 Aug 19, 2022
Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Hiroshechka Y 33 Dec 26, 2022
BirdCLEF 2021 - Birdcall Identification 4th place solution

BirdCLEF 2021 - Birdcall Identification 4th place solution My solution detail kaggle discussion Inference Notebook (best submission) Environment Use K

tattaka 42 Jan 2, 2023