a reimplementation of Holistically-Nested Edge Detection in PyTorch

Overview

pytorch-hed

This is a personal reimplementation of Holistically-Nested Edge Detection [1] using PyTorch. Should you be making use of this work, please cite the paper accordingly. Also, make sure to adhere to the licensing terms of the authors. Should you be making use of this particular implementation, please acknowledge it appropriately [2].

Paper

For the original version of this work, please see: https://github.com/s9xie/hed
For another reimplementation based on Caffe, please see: https://github.com/zeakey/hed

usage

To run it on your own image, use the following command. Please make sure to see their paper / the code for more details.

python run.py --model bsds500 --in ./images/sample.png --out ./out.png

I am afraid that this reimplementation is not entirely true to the original Caffe version, even though it utilizes the official weights. It achieves an ODS=0.774 versus the official ODS=0.780 on the BSDS500 dataset, evaluated using this code. Please feel free to contribute to this repository by submitting issues and pull requests.

comparison

Comparison

references

[1]  @inproceedings{Xie_ICCV_2015,
         author = {Saining Xie and Zhuowen Tu},
         title = {Holistically-Nested Edge Detection},
         booktitle = {IEEE International Conference on Computer Vision},
         year = {2015}
     }
[2]  @misc{pytorch-hed,
         author = {Simon Niklaus},
         title = {A Reimplementation of {HED} Using {PyTorch}},
         year = {2018},
         howpublished = {\url{https://github.com/sniklaus/pytorch-hed}}
    }
Comments
  • Switch to TorchVision

    Switch to TorchVision

    As seen in #22, using torchvision's transformations produces better edges as compared to the original solution.

    Unfortunately I am not able to test the ODS score with the repo you linked, but just from my untrained eyeballs it appears to be better. Will post more metrics later.

    opened by xiurobert 8
  • F-Score is terrible for individual images

    F-Score is terrible for individual images

    `Hello,

    I took images from the BSDS dataset, and individually analyzed the F-Scores. I observed that the result is pretty bad in general, and I am not sure what I'm doing wrong. I am suspecting a bug, but I'm not confident.

    Image vs ground truth: image

    HED output: image

    And of course, if the value of an individual cell is greater than 0.5, I round it off to 1 before estimating the F1 score. The image after rounding off looks like this:

    image

    Then, I calculate the F-Score using the following code:

    Here, the gt is the groundtruth and the pred array is the prediction image from HED.

    tp = np.sum(pred[np.where(pred == 1)] == gt[np.where(pred == 1)])
    fp = np.sum(pred[np.where(pred == 1)] != gt[np.where(pred == 1)])
    precision = tp / (tp + fp)
    
    # recall
    fn = np.sum(pred[np.where(pred == 0)] != gt[np.where(pred == 0)])
    recall = tp / (tp + fn)
    
    # f-value
    fvalue = 2 * precision * recall / (precision + recall)
    

    I get a value (F1 score) of 0.1 to 0.2. The paper and the Github descriptions state much higher values (>0.7). Can anyone guide me regarding what could be going wrong? Since I am using the same metric and the same score, it is a little tricky.

    Any help is appreciated.

    opened by Arka161 3
  • about pre-trained model

    about pre-trained model

    can you tell me how to open network-bsds500.pytorch(i.e.the file download use download.bash), i use .txt or .py the file will change to garbled。so,can you provide .pth file,or tell me how to open it?

    opened by tarxs 3
  • RuntimeError: shape '[1, 3, 320, 480]' is invalid for input of size 614400

    RuntimeError: shape '[1, 3, 320, 480]' is invalid for input of size 614400

    Traceback (most recent call last): File "run.py", line 153, in <module> tenOutput = estimate(tenInput) File "run.py", line 145, in estimate return netNetwork(tenInput.cuda().view(1, 3, intHeight, intWidth))[0, :, :, :].cpu() RuntimeError: shape '[1, 3, 320, 480]' is invalid for input of size 614400 I tried to test my own image, but I got this error. The difference between sample.png and my own image.png is the bit depth. Mine is 32 bit depth.

    opened by tkamkb 2
  • RGB Normalization problem

    RGB Normalization problem

    Hi, I noticed that in the forward function it normalizes the input as

    tenBlue = (tenInput[:, 0:1, :, :] * 255.0) - 104.00698793
    tenGreen = (tenInput[:, 1:2, :, :] * 255.0) - 116.66876762 
    tenRed = (tenInput[:, 2:3, :, :] * 255.0) - 122.67891434
    

    But when I noramlized it by ImageNet RGB mean[0.485, 0.456, 0.406] as follows:

    tenBlue = tenInput[:, 0:1, :, :]  - 0.406
    tenGreen = tenInput[:, 1:2, :, :] - 0.456
    tenRed = tenInput[:, 2:3, :, :] - 0.485
    

    The output become blurry: out2 I feel very confused about this cause it seems to be same normalization process

    opened by linhl97 2
  • Using HED as a term of feature-based loss

    Using HED as a term of feature-based loss

    Hi sniklaus, Thanks very much for u excellent implementation. There is a qusetion that I'd like to ask you for advice. Have you ever used HED's feature map as a supervised term, just similar to VGG loss? It would be greatly appreciated if you could offer me any suggestions! Best, Melon

    opened by MeLonJ10 2
  • problem of the pretrained model

    problem of the pretrained model

    HI ,good morning. So sorry to bother you . There are something wrong with the pretrained network as follows .

    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly) RuntimeError: unexpected EOF, expected 4922858 more bytes. The file might be corrupted.

    image

    opened by hello-trouble 2
  • bsds500 weights unavailable

    bsds500 weights unavailable

    Hey,

    I just wanted to run your re-implementation of the pytorch HED model and the weights url http://content.sniklaus.com/github/pytorch-hed/network-bsds500 seems to point nowhere.

    Keep up the good work!

    opened by tahesse 1
  • Multiple images as input

    Multiple images as input

    Hello, @sniklaus Thanks for your contribution! Is there a way to generate edges from multiple images under a specific directory with your run.py? It seemed that your run.py can only process a single image as input so far. Thanks.

    opened by JamesLee789 1
  • AttributeError: 'Tensor' object has no attribute 'clip'

    AttributeError: 'Tensor' object has no attribute 'clip'

    Hi there, it seemed that your latest commit(https://github.com/sniklaus/pytorch-hed/commit/5ef88c378053e0c865f4a60eb69db4c7f2714e8e) didn't work well with PyTorch 1.6? After python run.py, I got the following error,

    Traceback (most recent call last): File "run.py", line 154, in PIL.Image.fromarray((tenOutput.clip(0.0, 1.0).numpy().transpose(1, 2, 0)[:, :, 0] * 255.0).astype(numpy.uint8)).save(arguments_strOut) AttributeError: 'Tensor' object has no attribute 'clip'

    When I changed the clip back to clamp, it worked. Could you be so kind to provide me with any clues on this? Thanks.

    opened by mingyanglei 1
  • I am trying to run the command

    I am trying to run the command "python run.py --model bsds500 --in ./images/sample.png --out ./out.png" and I am getting the error:

    Traceback (most recent call last): File "run.py", line 152, in tenOutput = estimate(tenInput) File "run.py", line 135, in estimate netNetwork = Network().eval() File "run.py", line 95, in init self.load_state_dict({ strKey.replace('module', 'net'): tenWeight for strKey, tenWeight in torch.hub.load_state_dict_from_url(url='http://content.sniklaus.com/github/pytorch-hed/network-' + arguments_strModel + '.pytorch', file_name='hed-' + arguments_strModel).items() }) TypeError: load_state_dict_from_url() got an unexpected keyword argument 'file_name'

    opened by chaitanyaspatil 1
Owner
Simon Niklaus
Research Scientist at Adobe
Simon Niklaus
A PyTorch Implementation of the Luna: Linear Unified Nested Attention

Unofficial PyTorch implementation of Luna: Linear Unified Nested Attention The quadratic computational and memory complexities of the Transformer’s at

Soohwan Kim 32 Nov 7, 2022
text_recognition_toolbox: The reimplementation of a series of classical scene text recognition papers with Pytorch in a uniform way.

text recognition toolbox 1. 项目介绍 该项目是基于pytorch深度学习框架,以统一的改写方式实现了以下6篇经典的文字识别论文,论文的详情如下。该项目会持续进行更新,欢迎大家提出问题以及对代码进行贡献。 模型 论文标题 发表年份 模型方法划分 CRNN 《An End-t

null 168 Dec 24, 2022
PyTorch reimplementation of minimal-hand (CVPR2020)

Minimal Hand Pytorch Unofficial PyTorch reimplementation of minimal-hand (CVPR2020). you can also find in youtube or bilibili bare hand youtube or bil

Hao Meng 228 Dec 29, 2022
PyTorch reimplementation of the paper Involution: Inverting the Inherence of Convolution for Visual Recognition [CVPR 2021].

Involution: Inverting the Inherence of Convolution for Visual Recognition Unofficial PyTorch reimplementation of the paper Involution: Inverting the I

Christoph Reich 100 Dec 1, 2022
PyTorch reimplementation of hand-biomechanical-constraints (ECCV2020)

Hand Biomechanical Constraints Pytorch Unofficial PyTorch reimplementation of Hand-Biomechanical-Constraints (ECCV2020). This project reimplement foll

Hao Meng 59 Dec 20, 2022
A PyTorch Reimplementation of TecoGAN: Temporally Coherent GAN for Video Super-Resolution

TecoGAN-PyTorch Introduction This is a PyTorch reimplementation of TecoGAN: Temporally Coherent GAN for Video Super-Resolution (VSR). Please refer to

null 165 Dec 17, 2022
a reimplementation of Optical Flow Estimation using a Spatial Pyramid Network in PyTorch

pytorch-spynet This is a personal reimplementation of SPyNet [1] using PyTorch. Should you be making use of this work, please cite the paper according

Simon Niklaus 269 Jan 2, 2023
PyTorch reimplementation of REALM and ORQA

PyTorch reimplementation of REALM and ORQA

Li-Huai (Allan) Lin 17 Aug 20, 2022
a reimplementation of UnFlow in PyTorch that matches the official TensorFlow version

pytorch-unflow This is a personal reimplementation of UnFlow [1] using PyTorch. Should you be making use of this work, please cite the paper according

Simon Niklaus 134 Nov 20, 2022
a reimplementation of LiteFlowNet in PyTorch that matches the official Caffe version

pytorch-liteflownet This is a personal reimplementation of LiteFlowNet [1] using PyTorch. Should you be making use of this work, please cite the paper

Simon Niklaus 365 Dec 31, 2022
Pytorch reimplementation of PSM-Net: "Pyramid Stereo Matching Network"

This is a Pytorch Lightning version PSMNet which is based on JiaRenChang/PSMNet. use python main.py to start training. PSM-Net Pytorch reimplementatio

XIAOTIAN LIU 1 Nov 25, 2021
Unofficial PyTorch reimplementation of the paper Swin Transformer V2: Scaling Up Capacity and Resolution

PyTorch reimplementation of the paper Swin Transformer V2: Scaling Up Capacity and Resolution [arXiv 2021].

Christoph Reich 122 Dec 12, 2022
PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

Christoph Reich 10 Jan 2, 2023
Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Pixel Difference Convolution This repository contains the PyTorch implementation for "Pixel Difference Networks for Efficient Edge Detection" by Zhuo

Alex 236 Dec 21, 2022
pcnaDeep integrates cutting-edge detection techniques with tracking and cell cycle resolving models.

pcnaDeep: a deep-learning based single-cell cycle profiler with PCNA signal Welcome! pcnaDeep integrates cutting-edge detection techniques with tracki

ChanLab 8 Oct 18, 2022
Implementation for the paper 'YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs'

YOLO-ReT This is the original implementation of the paper: YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs. Prakhar Ganesh, Ya

null 69 Oct 19, 2022
CenterFace(size of 7.3MB) is a practical anchor-free face detection and alignment method for edge devices.

CenterFace Introduce CenterFace(size of 7.3MB) is a practical anchor-free face detection and alignment method for edge devices. Recent Update 2019.09.

StarClouds 1.2k Dec 21, 2022
A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

A Light and Fast Face Detector for Edge Devices Big News: LFD, which is a big update of LFFD, now is released (2021.03.09). It is strongly recommended

YonghaoHe 1.3k Dec 25, 2022
Structured Edge Detection Toolbox

################################################################### # # # Structure

Piotr Dollar 779 Jan 2, 2023