A Keras implementation of YOLOv3 (Tensorflow backend)

Overview

keras-yolo3

license

Introduction

A Keras implementation of YOLOv3 (Tensorflow backend) inspired by allanzelener/YAD2K.


Quick Start

  1. Download YOLOv3 weights from YOLO website.
  2. Convert the Darknet YOLO model to a Keras model.
  3. Run YOLO detection.
wget https://pjreddie.com/media/files/yolov3.weights
python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5
python yolo_video.py [OPTIONS...] --image, for image detection mode, OR
python yolo_video.py [video_path] [output_path (optional)]

For Tiny YOLOv3, just do in a similar way, just specify model path and anchor path with --model model_file and --anchors anchor_file.

Usage

Use --help to see usage of yolo_video.py:

usage: yolo_video.py [-h] [--model MODEL] [--anchors ANCHORS]
                     [--classes CLASSES] [--gpu_num GPU_NUM] [--image]
                     [--input] [--output]

positional arguments:
  --input        Video input path
  --output       Video output path

optional arguments:
  -h, --help         show this help message and exit
  --model MODEL      path to model weight file, default model_data/yolo.h5
  --anchors ANCHORS  path to anchor definitions, default
                     model_data/yolo_anchors.txt
  --classes CLASSES  path to class definitions, default
                     model_data/coco_classes.txt
  --gpu_num GPU_NUM  Number of GPU to use, default 1
  --image            Image detection mode, will ignore all positional arguments

  1. MultiGPU usage: use --gpu_num N to use N GPUs. It is passed to the Keras multi_gpu_model().

Training

  1. Generate your own annotation file and class names file.
    One row for one image;
    Row format: image_file_path box1 box2 ... boxN;
    Box format: x_min,y_min,x_max,y_max,class_id (no space).
    For VOC dataset, try python voc_annotation.py
    Here is an example:

    path/to/img1.jpg 50,100,150,200,0 30,50,200,120,3
    path/to/img2.jpg 120,300,250,600,2
    ...
    
  2. Make sure you have run python convert.py -w yolov3.cfg yolov3.weights model_data/yolo_weights.h5
    The file model_data/yolo_weights.h5 is used to load pretrained weights.

  3. Modify train.py and start training.
    python train.py
    Use your trained weights or checkpoint weights with command line option --model model_file when using yolo_video.py Remember to modify class path or anchor path, with --classes class_file and --anchors anchor_file.

If you want to use original pretrained weights for YOLOv3:
1. wget https://pjreddie.com/media/files/darknet53.conv.74
2. rename it as darknet53.weights
3. python convert.py -w darknet53.cfg darknet53.weights model_data/darknet53_weights.h5
4. use model_data/darknet53_weights.h5 in train.py


Some issues to know

  1. The test environment is

    • Python 3.5.2
    • Keras 2.1.5
    • tensorflow 1.6.0
  2. Default anchors are used. If you use your own anchors, probably some changes are needed.

  3. The inference result is not totally the same as Darknet but the difference is small.

  4. The speed is slower than Darknet. Replacing PIL with opencv may help a little.

  5. Always load pretrained weights and freeze layers in the first stage of training. Or try Darknet training. It's OK if there is a mismatch warning.

  6. The training strategy is for reference only. Adjust it according to your dataset and your goal. And add further strategy if needed.

  7. For speeding up the training process with frozen layers train_bottleneck.py can be used. It will compute the bottleneck features of the frozen model first and then only trains the last layers. This makes training on CPU possible in a reasonable time. See this for more information on bottleneck features.

Comments
  • failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED

    failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED

    When I train voc data, the error happened. My GPU is RTX2080 8G * 2,tensorflow-gpu:1.12,keras2.2.4

    Epoch 1/50 2019-01-28 00:16:00.441512: E tensorflow/stream_executor/cuda/cuda_blas.cc:652] failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED Traceback (most recent call last): File "train.py", line 192, in <module> _main(annotation_path=anno) File "train.py", line 65, in _main callbacks=[logging, checkpoint]) File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1418, in fit_generator initial_epoch=initial_epoch) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py", line 217, in fit_generator class_weight=class_weight) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1217, in train_on_batch outputs = self.train_function(ins) File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 2715, in __call__ return self._call(inputs) File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 2675, in _call fetched = self._callable_fn(*array_vals) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1439, in __call__ run_metadata_ptr) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InternalError: Blas SGEMM launch failed : m=346112, n=32, k=64 [[{{node conv2d_3/convolution}} = Conv2D[T=DT_FLOAT, _class=["loc:@batch_normalization_3/cond/FusedBatchNorm/Switch"], data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](leaky_re_lu_2/LeakyRelu, conv2d_3/kernel/read)]] [[{{node yolo_loss/while_1/LoopCond/_2963}} = _HostRecv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_6607_yolo_loss/while_1/LoopCond", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopyolo_loss/while_1/strided_slice_1/stack_2/_2805)]]

    opened by ouening 32
  • NameError: name 'yolo_head' is not defined

    NameError: name 'yolo_head' is not defined

    When I save weights during training using your original code, I got:

    InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 12675 values, but the requested shape requires a multiple of 3042
    	 [[Node: Reshape_3 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_59/BiasAdd, Reshape_3/shape)]]
    

    Does this mean I should save model instead of weights? May I ask why in your code "model.save_weights" is used instead of "model.save_path"?

    When I saved checkpoints in train.py as keras model, I wasn't able to use it in yolo.py using load_model.

    checkpoint = ModelCheckpoint(log_dir + "checkpoint.h5", monitor='val_loss', save_best_only=True)
    
    history = model.fit([image_data, *y_true],
              np.zeros(len(image_data)),
              validation_split=.1,
              batch_size=Batch_Size,
              epochs=10000,
              callbacks=[checkpoint])
    

    When I use yolo.py to test the trained model on images, model is loaded using

    self.yolo_model = load_model(model_path, compile=False)
    

    I got this error:

    2018-05-10 05:47:25.060203: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
    Traceback (most recent call last):
      File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 229, in <module>
        detect_img(YOLO())
      File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 58, in __init__
        self.boxes, self.scores, self.classes = self.generate()
      File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 80, in generate
        self.yolo_model = load_model(model_path, compile=False)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/models.py", line 243, in load_model
        model = model_from_config(model_config, custom_objects=custom_objects)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/models.py", line 317, in model_from_config
        return layer_module.deserialize(config, custom_objects=custom_objects)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
        printable_module_name='layer')
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 144, in deserialize_keras_object
        list(custom_objects.items())))
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 2524, in from_config
        process_node(layer, node_data)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 2483, in process_node
        layer(input_tensors, **kwargs)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
        output = self.call(inputs, **kwargs)
      File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/layers/core.py", line 663, in call
        return self.function(inputs, **arguments)
      File "/home/jinz1/Jin/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo3/model.py", line 347, in yolo_loss
    NameError: name 'yolo_head' is not defined
    

    Thank you very much for your help and for creating this repo!

    opened by JudyMRSD 32
  • What's annotation_path = 'train.txt'?

    What's annotation_path = 'train.txt'?

    I get this error: minh@minh-Aspire-A515-51G:~/keras-yolo3$ python train.py /home/minh/.local/lib/python3.5/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters Using TensorFlow backend. Traceback (most recent call last): File "train.py", line 153, in _main() File "train.py", line 31, in _main input_shape, max_boxes=100, load_previous=True) File "train.py", line 76, in get_training_data image = Image.open(filename) File "/home/minh/.local/lib/python3.5/site-packages/PIL/Image.py", line 2548, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: '000012\n'

    I don't know where is the file 'train.txt' and i got this file in VOCdevkit/VOC2007/ImageSets/Main? Can you help me?

    opened by maiminh1996 24
  • Has anyone tried to use C++ to call a trained model (model.h5)?

    Has anyone tried to use C++ to call a trained model (model.h5)?

    I added in train.py a model.save(log_dir + 'model.h5') next to the model.save_weights statement. I use the method gived by amai to convert the model.h5 to model.pb file, but I get the error that yolo_head is not defined in yolo_loss.

    opened by zhoujinhai 17
  • How to load the self-trained model to display the feature maps?

    How to load the self-trained model to display the feature maps?

    when I use the "load_model()", I get the following error:

    from keras.models import load_model,Model
    base_model = load_model('yolo_model.h5') #yolo_model.h5 is the model trained on my own 
                                                                           #dataset.
    print(base_model.summary())
    

    .... File "C:\Users\ww\Desktop\PycharmProjects\YOLO\keras-yolo3\yolo3\model.py", line 459, in yolo_loss grid, raw_pred, pred_xy, pred_wh = yolo_head(yolo_outputs[l], NameError: name 'yolo_head' is not defined

    opened by Bin-913 16
  • How can I convert my own model from darknet to keras?

    How can I convert my own model from darknet to keras?

    I used my own data to train the model detect the specific class, and the model run well in darknet. But when I try to convert the model , I can not detect any object. I use my own cfg and my own .weight file when convert the model, and I change the path of model path and class path. Is there any place need to change?

    opened by LiangHao92 16
  • For new datasets

    For new datasets

    I want to use the code to detect the ID card. Right now I have some data collected and labeled. Is it OK I run train.py with only my dataset, which only has 1 class: id card?

    opened by jindongwang 15
  • Mismatch between model and given anchor and class sizes

    Mismatch between model and given anchor and class sizes

    My error occur when i use model tiny yolo? Error in yolo.py

    ---> 35         self.boxes, self.scores, self.classes = self.generate()
         36 
         37     def _get_class(self):
    
    ~/Documents/Object_Detection/keras-yolo3/yolo.py in generate(self)
         66             assert self.yolo_model.layers[-1].output_shape[-1] == \
         67                 num_anchors/len(self.yolo_model.output) * (num_classes + 5), \
    ---> 68                 'Mismatch between model and given anchor and class sizes'
         69 
         70         print('{} model, anchors, and classes loaded.'.format(model_path))
    
    AssertionError: Mismatch between model and given anchor and class sizes
    
    opened by damvantai 13
  • Keras yolo3 model output layers objective in tensorflow serving

    Keras yolo3 model output layers objective in tensorflow serving

    I'm trying to use yolo3 with tensorflow serving and I'm having hard time converting returned response values.

    I exported Keras model for TF Serving and as I can see there are three outputs available in keras model as follow:

    >> yolo_keras_model.output
    [<tf.Tensor 'conv2d_59_1/BiasAdd:0' shape=(?, ?, ?, 255) dtype=float32>,
     <tf.Tensor 'conv2d_67_1/BiasAdd:0' shape=(?, ?, ?, 255) dtype=float32>,
     <tf.Tensor 'conv2d_75_1/BiasAdd:0' shape=(?, ?, ?, 255) dtype=float32>]
    

    ... I exported the model with following signature:

    signature = predict_signature_def(
        inputs={'inputs': yolo_keras_model.input}, 
        outputs={
                'conv2d_59': yolo_keras_model.output[0],
                'conv2d_67': yolo_keras_model.output[1],
                'conv2d_75': yolo_keras_model.output[2]
        }
    

    I'm getting following response in tensroflow serving client script:

    response = stub.Predict(request, 60.0)
    conv2d_59 = np.squeeze(tf.contrib.util.make_ndarray(response.outputs['conv2d_59']))
    conv2d_67 = np.squeeze(tf.contrib.util.make_ndarray(response.outputs['conv2d_67']))
    conv2d_75 = np.squeeze(tf.contrib.util.make_ndarray(response.outputs['conv2d_67']))
    
    >> conv2d_59.shape, conv2d_67.shape, conv2d_75.shape
    ((15, 25, 255), (30, 50, 255), (60, 100, 255))
    

    The confusion here is, how should I interpret each outputs? Are these values correlate with box, score, classes values?

    opened by spate141 13
  • K.sigmoid in xy_loss with binary cross entropy

    K.sigmoid in xy_loss with binary cross entropy

    Hi,

    I see the yolo head in model.py line 140

    feats[...,:2] is after sigmoid function https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L140

    But in the loss function raw_true_xy is just inverse of line 140 without sigmoid https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L380 ... In xy_loss ,raw_pred[...,0:2] also didn't have sigmoid function https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L399

    Why in yolo_head need sigmoid but train didn't need

    Did i miss something?

    Thanks!

    opened by franktpmvu 11
  • about train.py

    about train.py

    hello, when i run python train.py, it returns error as: type error,load_weights() got 2 argument 'skip_mismatch' argument error, it seems like 'skip_mismatch'returns the argument error,whats the problem, my environment setting as keras version:2.1.5, pls help,tks,by simon

    opened by simonhandsome 11
  • Questions about train model

    Questions about train model

    1. Suppose there's a picture with size (1200,800) and objectA is in (x,y,w,h)=(300,300,200,100), do I have to change the ratio to (416,416) or just input (300,300,200,100)?

    2. whether the output of (x,y,w,z,....) is in (416,416) or the original size (1200,800) ?

    thanks

    opened by wielh 0
  • Error when Data augmentation removed! Need help

    Error when Data augmentation removed! Need help

    ---> 66 y_true[l][b, j, i, k, 0:4] = true_boxes[b,t, 0:4] 67 y_true[l][b, j, i, k, 4] = 1 68 y_true[l][b, j, i, k, 5+c] = 1

    IndexError: index 13 is out of bounds for axis 2 with size 13

    opened by shamsudheen-marakkar 0
  • Training

    Training

    Hey guys, I've run all the quick start commands and I'm now at the training section Am I to just,

    1. Run python voc_annotation.py (I have no custom datasets)
    2. And then run python convert.py -w yolov3.cfg yolov3.weights model_data/yolo_weights.h5
    3. And then finally run train.py? Is this going to work please?
    opened by btopii32 3
  • why the 2nd element of the yield in data_generator is of size equal to batch_size?

    why the 2nd element of the yield in data_generator is of size equal to batch_size?

    Following is the link to the exact line of code. I just want to understand the theory behind it. https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/train.py#L182

    opened by sonalrpatel 1
Owner
null
Cascaded Pyramid Network (CPN) based on Keras (Tensorflow backend)

ML2 Takehome Project Reimplementing the paper: Cascaded Pyramid Network for Multi-Person Pose Estimation Dataset The model uses the COCO dataset which

Vo Van Tu 1 Nov 22, 2021
🔥 TensorFlow Code for technical report: "YOLOv3: An Incremental Improvement"

?? Are you looking for a new YOLOv3 implemented by TF2.0 ? If you hate the fucking tensorflow1.x very much, no worries! I have implemented a new YOLOv

null 3.6k Dec 26, 2022
Minimal PyTorch implementation of YOLOv3

A minimal PyTorch implementation of YOLOv3, with support for training, inference and evaluation.

Erik Linder-Norén 6.9k Dec 29, 2022
Yolov3 pytorch implementation

YOLOV3 Pytorch实现 在bubbliiing大佬代码的基础上进行了修改,添加了部分注释。 预训练模型 预训练模型来源于bubbliiing。 链接:https://pan.baidu.com/s/1ncREw6Na9ycZptdxiVMApw 提取码:appk 训练自己的数据集 按照VO

null 4 Aug 27, 2022
This is an implementation of Googles Yogi-Optimizer in Keras (tf.keras)

Yogi-Optimizer_Keras This is an implementation of Googles Yogi-Optimizer in Keras (tf.keras) The NeurIPS-Paper can be found here: http://papers.nips.c

null 14 Sep 13, 2022
Keras udrl - Keras implementation of Upside Down Reinforcement Learning

keras_udrl Keras implementation of Upside Down Reinforcement Learning This is me

Eder Santana 7 Jan 24, 2022
A high-performance anchor-free YOLO. Exceeding yolov3~v5 with ONNX, TensorRT, NCNN, and Openvino supported.

YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and industrial communities. For more details, please refer to our report on Arxiv.

null 7.7k Jan 6, 2023
YOLOv3 in PyTorch > ONNX > CoreML > TFLite

This repository represents Ultralytics open-source research into future object detection methods, and incorporates lessons learned and best practices

Ultralytics 9.3k Jan 7, 2023
YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with ONNX, TensorRT, ncnn, and OpenVINO supported.

Introduction YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and ind

null 7.7k Jan 3, 2023
Train a state-of-the-art yolov3 object detector from scratch!

TrainYourOwnYOLO: Building a Custom Object Detector from Scratch This repo let's you train a custom image detector using the state-of-the-art YOLOv3 c

AntonMu 616 Jan 8, 2023
Dataset para entrenamiento de yoloV3 para 4 clases

Deteccion de objetos en video Este repo basado en el proyecto PyTorch YOLOv3 para correr detección de objetos sobre video. Construí sobre este proyect

null 1 Nov 1, 2021
This project deals with the detection of skin lesions within the ISICs dataset using YOLOv3 Object Detection with Darknet.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Skin Lesion detection using YOLO This project deal

Lalith Veerabhadrappa Badiger 1 Nov 22, 2021
Object Detection with YOLOv3

Object Detection with YOLOv3 Bu projede YOLOv3-608 modeli kullanılmıştır. Requirements Python 3.8 OpenCV Numpy Documentation Yolo ile ilgili detaylı b

Ayşe Konuş 0 Mar 27, 2022
Multiple custom object count and detection using YOLOv3-Tiny method

Electronic-Component-YOLOv3 Introduce This project created to detect, count, and recognize multiple custom object using YOLOv3-Tiny method. The target

Derwin Mahardika 2 Nov 14, 2022
Torchyolo - Yolov3 ve Yolov4 modellerin Pytorch uygulamasıdır

TORCHYOLO : Yolo Modellerin Pytorch Uygulaması Yapılacaklar: Yolov3 model.py ve

Kadir Nar 3 Aug 22, 2022
Example-custom-ml-block-keras - Custom Keras ML block example for Edge Impulse

Custom Keras ML block example for Edge Impulse This repository is an example on

Edge Impulse 8 Nov 2, 2022
Classification models 1D Zoo - Keras and TF.Keras

Classification models 1D Zoo - Keras and TF.Keras This repository contains 1D variants of popular CNN models for classification like ResNets, DenseNet

Roman Solovyev 12 Jan 6, 2023
Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Fastformer-Keras Unofficial Tensorflow-Keras implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Tensorflo

Yam Peleg 10 Jan 30, 2022
Implementation of ConvMixer in TensorFlow and Keras

ConvMixer ConvMixer, an extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in that it operates directly on

Sayan Nath 8 Oct 3, 2022