Open-source Monocular Python HawkEye for Tennis

Overview

Tennis Tracking 🎾

Objectives

  • Track the ball
  • Detect court lines
  • Detect the players

To track the ball we used TrackNet - deep learning network for tracking high-speed objects. For players detection yolov3 was used.

Example using sample videos

Input Output
input_img1 output_img1
input_img2 output_img2
input_img3 output_img3

How to run

This project requires compatible GPU to install tensorflow, you can run it on your local machine in case you have one or use Google Colaboratory with Runtime Type changed to GPU.

  1. Clone this repository
  2. git clone https://github.com/ArtLabss/tennis-tracking
    
  3. Download yolov3 weights (237 MB) from here and add it to your Yolov3 folder.
  4. Install the requirements using pip
  5. pip install -r requirements.txt
  6. Run the following command in the command line
  7. python predict_video.py --input_video_path=VideoInput/video_input3.mp4 --output_video_path=VideoOutput/video_output.mp4 --minimap=0
  8. If you are using Google Colab upload all the files to Google Drive
  9. Create a Google Colaboratory Notebook in the same directory as predict_video.py and connect it to Google drive
  10. from google.colab import drive
    drive.mount('/content/drive')
  11. Change the working directory to the one where the Colab Notebook and predict_video.py are. In my case,
  12. import os 
    os.chdir('MyDrive/Colab Notebooks/tennis-tracking')
  13. Install the requirements
  14. !pip install -r requirements.txt
  15. Inside the notebook run predict_video.py
  16.  !python3 predict_video.py --input_video_path=VideoInput/video_input3.mp4 --output_video_path=VideoOutput/video_output.mp4 --minimap=0
    

    After the compilation is completed, a new video will be created in VideoOutput folder if --minimap was set 0, if --minimap=1 three videos will be created: video of the game, video of minimap and a combined video of both

    P.S. If you stumble upon an error or have any questions feel free to open a new Issue

What's new?

  • Court line detection improved
  • Player detection improved
  • The algorithm now works practically with any court colors
  • Faster algorithm
  • Dynamic Mini-Map with players and ball added, to activate use argument --minimap
--minimap=0 --minimap=1
input_img1 output_img1

Further Developments

  • Improve line detection of the court and remove overlapping lines
  • Algorithm fails to detect players when the court colors aren't similar to the sample video
  • Don't detect the ballboys/ballgirls
  • Don't contour the banners
  • Detect players on videos with different angles
  • Find the coordinates of the ball touching the court and display them
  • Code Optimization
  • Dynamic court mini-map with players and the ball

Current Drawbacks

  • Slow algorithms (to process 15 seconds video (6.1 Mb) it takes 28 minutes 16 minutes)
  • Algorithm works only on official match videos

References

- Yu-Chuan Huang, "TrackNet: Tennis Ball Tracking from Broadcast Video by Deep Learning Networks," Master Thesis, advised by Tsì-Uí İk and Guan-Hua Huang, National Chiao Tung University, Taiwan, April 2018. - Yu-Chuan Huang, I-No Liao, Ching-Hsuan Chen, Tsì-Uí İk, and Wen-Chih Peng, "TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications," in the IEEE International Workshop of Content-Aware Video Analysis (CAVA 2019) in conjunction with the 16th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS 2019), 18-21 September 2019, Taipei, Taiwan. - Joseph Redmon, Ali Farhadi, "YOLOv3: An Incremental Improvement", University of Washington, https://arxiv.org/pdf/1804.02767.pdf
Comments
  • cant' load weights and predict

    cant' load weights and predict

    1.The model will have the following problems when loading parameters: 2022-04-04 12:43:19.257086: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open WeightsTracknet\model.1: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

    2.When I change model.1 to model.h5, the above error message will disappear, but the model will break when predicting.

    I am sorry for taking your time and hope you can help!

    bug 
    opened by nlpCSir 7
  • Error at step 8 of How to Run

    Error at step 8 of How to Run

    Describe the bug When trying to run the code in Google Colab, I get an error when I am at step 8: !pip install filterpy sktime

    When I do this, I get the following errors message: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. yellowbrick 1.3.post1 requires numpy<1.20,>=1.16.0, but you have numpy 1.21.5 which is incompatible. datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible. albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible. Successfully installed deprecated-1.2.13 filterpy-1.4.5 llvmlite-0.38.0 numba-0.55.1 numpy-1.21.5 sktime-0.10.0 statsmodels-0.13.1 WARNING: The following packages were previously imported in this runtime: [numpy] You must restart the runtime in order to use newly installed versions.

    When I restart the runtime, it does not help. When I try to run predict_video.py, I get the following error:

    2022-02-08 15:57:12.483066: W tensorflow/core/util/tensor_slice_reader.cc:96] Could not open WeightsTracknet/model.1: DATA_LOSS: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 12 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'

    I do not know if the second error is the cause of the first error, but I would love to be able to successfully implement the interesting algorithm. I am looking forward to your reply.

    Desktop (please complete the following information):

    • OS: Windows
    • Browser chrome
    • Version [e.g. 22]

    Smartphone (please complete the following information):

    • Device: [e.g. iPhone6]
    • OS: [e.g. iOS8.1]
    • Browser [e.g. stock browser, safari]
    • Version [e.g. 22]

    Additional context Add any other context about the problem here.

    bug 
    opened by PaulvanderWolde 7
  • How did you decide on x,y and velocity of ball to predict bounces?

    How did you decide on x,y and velocity of ball to predict bounces?

    To predict bounce points machine learning library for time series sktime was used. Specifically, TimeSeriesForestClassifier was trained on 3 variables: x, y coordinates of the ball and V for velocity (V2-V1/t2-t1). Additional context Add any other context or screenshots about the feature request here.

    I want to try to improve the predictive accuracy but don't want to reinvent the wheel if you can share some info on what you tried and how you decided on these three factors.

    question 
    opened by nyck33 6
  • -215:Assertion failed) count > 0 in function 'fitLine2D_wods'

    -215:Assertion failed) count > 0 in function 'fitLine2D_wods'

    Hi,

    First of all, this is great work!

    This is the error I get when I am trying to run on a tennis video. After printing model summary, this follows.

    BOXES [] BOXES [] BOXES [array([166.0948 , 679.272 , 195.11845, 705.8905 ], dtype=float32)] BIGGEST [166. 679. 195. 706.] CAMERA ... Court tracking failed, adding 5 pixels to dist CAMERA ... Court tracking failed, adding 5 pixels to dist CAMERA ... Court tracking failed, adding 5 pixels to dist Traceback (most recent call last): File "predict_video.py", line 113, in lines = court_detector.track_court(frame) File "/content/drive/My Drive/TennisTracking/NewTracking/tennis-tracking/court_detector.py", line 445, in track_court return self.track_court(frame) File "/content/drive/My Drive/TennisTracking/NewTracking/tennis-tracking/court_detector.py", line 445, in track_court return self.track_court(frame) File "/content/drive/My Drive/TennisTracking/NewTracking/tennis-tracking/court_detector.py", line 445, in track_court return self.track_court(frame) File "/content/drive/My Drive/TennisTracking/NewTracking/tennis-tracking/court_detector.py", line 421, in track_court [vx, vy, x, y] = cv2.fitLine(new_points, cv2.DIST_L2, 0, 0.01, 0.01) cv2.error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/linefit.cpp:50: error: (-215:Assertion failed) count > 0 in function 'fitLine2D_wods'

    bug documentation question 
    opened by SaiThejeshwar 5
  • AssertionError: X must have unique column indices, but found Int64Index([0, 0, 0], dtype='int64')

    AssertionError: X must have unique column indices, but found Int64Index([0, 0, 0], dtype='int64')

    Hi I am getting this error when I am trying to detect the bounce. Here is my code for detecting bounce.

    bounce=1
    if bounce == 1:
      # Predicting Bounces 
      test_df = pd.DataFrame({'x': [coord[0] for coord in xy[:-1]], 'y':[coord[1] for coord in xy[:-1]], 'V': V})
    
      # df.shift
      for i in range(20, 0, -1): 
        test_df[f'lagX_{i}'] = test_df['x'].shift(i, fill_value=0)
      for i in range(20, 0, -1): 
        test_df[f'lagY_{i}'] = test_df['y'].shift(i, fill_value=0)
      for i in range(20, 0, -1): 
        test_df[f'lagV_{i}'] = test_df['V'].shift(i, fill_value=0)
    
      test_df.drop(['x', 'y', 'V'], 1, inplace=True)
    
      Xs = test_df[['lagX_20', 'lagX_19', 'lagX_18', 'lagX_17', 'lagX_16',
            'lagX_15', 'lagX_14', 'lagX_13', 'lagX_12', 'lagX_11', 'lagX_10',
            'lagX_9', 'lagX_8', 'lagX_7', 'lagX_6', 'lagX_5', 'lagX_4', 'lagX_3',
            'lagX_2', 'lagX_1']]
      Xs = from_2d_array_to_nested(Xs.to_numpy())
      Xs.columns = [1]
    
      Ys = test_df[['lagY_20', 'lagY_19', 'lagY_18', 'lagY_17',
            'lagY_16', 'lagY_15', 'lagY_14', 'lagY_13', 'lagY_12', 'lagY_11',
            'lagY_10', 'lagY_9', 'lagY_8', 'lagY_7', 'lagY_6', 'lagY_5', 'lagY_4',
            'lagY_3', 'lagY_2', 'lagY_1']]
      Ys = from_2d_array_to_nested(Ys.to_numpy())
      Ys.columns = [2]
    
      Vs = test_df[['lagV_20', 'lagV_19', 'lagV_18',
            'lagV_17', 'lagV_16', 'lagV_15', 'lagV_14', 'lagV_13', 'lagV_12',
            'lagV_11', 'lagV_10', 'lagV_9', 'lagV_8', 'lagV_7', 'lagV_6', 'lagV_5',
            'lagV_4', 'lagV_3', 'lagV_2', 'lagV_1']]
      Vs = from_2d_array_to_nested(Vs.to_numpy())
      Vs.columns = [3]
    
      X = pd.concat([Xs, Ys, Vs], 1)
    
      # load the pre-trained classifier  
      clf = load(open('clf.pkl', 'rb'))
    
    predcted = clf.predict(X)
    

    Following is the detailed error

    105   clf = load(open('clf.pkl', 'rb'))
        106 
    --> 107   predcted = clf.predict(X)
        108   idx = list(np.where(predcted == 1)[0])
        109   idx = np.array(idx) - 10
    
    5 frames
    [/usr/local/lib/python3.7/dist-packages/sktime/datatypes/_series/_check.py](https://localhost:8080/#) in check_pddataframe_series(obj, return_metadata, var_name)
         74     # check that columns are unique
         75     msg = f"{var_name} must have " f"unique column indices, but found {obj.columns}"
    ---> 76     assert obj.columns.is_unique, msg
         77 
         78     # check whether the time index is of valid type
    
    AssertionError: X must have unique column indices, but found Int64Index([0, 0, 0], dtype='int64')
    
    question 
    opened by madeel6233 3
  • Help: Failed to get convolution algorithm

    Help: Failed to get convolution algorithm

    2021-09-22 08:06:14.234343: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
    fps : 60
    2021-09-22 08:06:16.272909: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
    2021-09-22 08:06:16.294530: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.295305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
    pciBusID: 0000:00:04.0 name: Tesla K80 computeCapability: 3.7
    coreClock: 0.8235GHz coreCount: 13 deviceMemorySize: 11.17GiB deviceMemoryBandwidth: 223.96GiB/s
    2021-09-22 08:06:16.295393: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
    2021-09-22 08:06:16.299668: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
    2021-09-22 08:06:16.299764: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
    2021-09-22 08:06:16.301051: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
    2021-09-22 08:06:16.301799: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
    2021-09-22 08:06:16.305778: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11
    2021-09-22 08:06:16.306866: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
    2021-09-22 08:06:16.307181: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
    2021-09-22 08:06:16.307336: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.308216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.309021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
    2021-09-22 08:06:16.309402: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
    2021-09-22 08:06:16.309722: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.310485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
    pciBusID: 0000:00:04.0 name: Tesla K80 computeCapability: 3.7
    coreClock: 0.8235GHz coreCount: 13 deviceMemorySize: 11.17GiB deviceMemoryBandwidth: 223.96GiB/s
    2021-09-22 08:06:16.310635: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.311429: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.312205: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
    2021-09-22 08:06:16.312300: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
    2021-09-22 08:06:16.884681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
    2021-09-22 08:06:16.884740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
    2021-09-22 08:06:16.884767: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
    2021-09-22 08:06:16.885046: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.886096: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.887106: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2021-09-22 08:06:16.887864: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
    2021-09-22 08:06:16.887939: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10800 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
    layer24 output shape: 256 360 640
    Model: "model_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    input_1 (InputLayer)         [(None, 3, 360, 640)]     0         
    _________________________________________________________________
    conv2d (Conv2D)              (None, 64, 360, 640)      1792      
    _________________________________________________________________
    activation (Activation)      (None, 64, 360, 640)      0         
    _________________________________________________________________
    batch_normalization (BatchNo (None, 64, 360, 640)      2560      
    _________________________________________________________________
    conv2d_1 (Conv2D)            (None, 64, 360, 640)      36928     
    _________________________________________________________________
    activation_1 (Activation)    (None, 64, 360, 640)      0         
    _________________________________________________________________
    batch_normalization_1 (Batch (None, 64, 360, 640)      2560      
    _________________________________________________________________
    max_pooling2d (MaxPooling2D) (None, 64, 180, 320)      0         
    _________________________________________________________________
    conv2d_2 (Conv2D)            (None, 128, 180, 320)     73856     
    _________________________________________________________________
    activation_2 (Activation)    (None, 128, 180, 320)     0         
    _________________________________________________________________
    batch_normalization_2 (Batch (None, 128, 180, 320)     1280      
    _________________________________________________________________
    conv2d_3 (Conv2D)            (None, 128, 180, 320)     147584    
    _________________________________________________________________
    activation_3 (Activation)    (None, 128, 180, 320)     0         
    _________________________________________________________________
    batch_normalization_3 (Batch (None, 128, 180, 320)     1280      
    _________________________________________________________________
    max_pooling2d_1 (MaxPooling2 (None, 128, 90, 160)      0         
    _________________________________________________________________
    conv2d_4 (Conv2D)            (None, 256, 90, 160)      295168    
    _________________________________________________________________
    activation_4 (Activation)    (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_4 (Batch (None, 256, 90, 160)      640       
    _________________________________________________________________
    conv2d_5 (Conv2D)            (None, 256, 90, 160)      590080    
    _________________________________________________________________
    activation_5 (Activation)    (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_5 (Batch (None, 256, 90, 160)      640       
    _________________________________________________________________
    conv2d_6 (Conv2D)            (None, 256, 90, 160)      590080    
    _________________________________________________________________
    activation_6 (Activation)    (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_6 (Batch (None, 256, 90, 160)      640       
    _________________________________________________________________
    max_pooling2d_2 (MaxPooling2 (None, 256, 45, 80)       0         
    _________________________________________________________________
    conv2d_7 (Conv2D)            (None, 512, 45, 80)       1180160   
    _________________________________________________________________
    activation_7 (Activation)    (None, 512, 45, 80)       0         
    _________________________________________________________________
    batch_normalization_7 (Batch (None, 512, 45, 80)       320       
    _________________________________________________________________
    conv2d_8 (Conv2D)            (None, 512, 45, 80)       2359808   
    _________________________________________________________________
    activation_8 (Activation)    (None, 512, 45, 80)       0         
    _________________________________________________________________
    batch_normalization_8 (Batch (None, 512, 45, 80)       320       
    _________________________________________________________________
    conv2d_9 (Conv2D)            (None, 512, 45, 80)       2359808   
    _________________________________________________________________
    activation_9 (Activation)    (None, 512, 45, 80)       0         
    _________________________________________________________________
    batch_normalization_9 (Batch (None, 512, 45, 80)       320       
    _________________________________________________________________
    up_sampling2d (UpSampling2D) (None, 512, 90, 160)      0         
    _________________________________________________________________
    conv2d_10 (Conv2D)           (None, 256, 90, 160)      1179904   
    _________________________________________________________________
    activation_10 (Activation)   (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_10 (Batc (None, 256, 90, 160)      640       
    _________________________________________________________________
    conv2d_11 (Conv2D)           (None, 256, 90, 160)      590080    
    _________________________________________________________________
    activation_11 (Activation)   (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_11 (Batc (None, 256, 90, 160)      640       
    _________________________________________________________________
    conv2d_12 (Conv2D)           (None, 256, 90, 160)      590080    
    _________________________________________________________________
    activation_12 (Activation)   (None, 256, 90, 160)      0         
    _________________________________________________________________
    batch_normalization_12 (Batc (None, 256, 90, 160)      640       
    _________________________________________________________________
    up_sampling2d_1 (UpSampling2 (None, 256, 180, 320)     0         
    _________________________________________________________________
    conv2d_13 (Conv2D)           (None, 128, 180, 320)     295040    
    _________________________________________________________________
    activation_13 (Activation)   (None, 128, 180, 320)     0         
    _________________________________________________________________
    batch_normalization_13 (Batc (None, 128, 180, 320)     1280      
    _________________________________________________________________
    conv2d_14 (Conv2D)           (None, 128, 180, 320)     147584    
    _________________________________________________________________
    activation_14 (Activation)   (None, 128, 180, 320)     0         
    _________________________________________________________________
    batch_normalization_14 (Batc (None, 128, 180, 320)     1280      
    _________________________________________________________________
    up_sampling2d_2 (UpSampling2 (None, 128, 360, 640)     0         
    _________________________________________________________________
    conv2d_15 (Conv2D)           (None, 64, 360, 640)      73792     
    _________________________________________________________________
    activation_15 (Activation)   (None, 64, 360, 640)      0         
    _________________________________________________________________
    batch_normalization_15 (Batc (None, 64, 360, 640)      2560      
    _________________________________________________________________
    conv2d_16 (Conv2D)           (None, 64, 360, 640)      36928     
    _________________________________________________________________
    activation_16 (Activation)   (None, 64, 360, 640)      0         
    _________________________________________________________________
    batch_normalization_16 (Batc (None, 64, 360, 640)      2560      
    _________________________________________________________________
    conv2d_17 (Conv2D)           (None, 256, 360, 640)     147712    
    _________________________________________________________________
    activation_17 (Activation)   (None, 256, 360, 640)     0         
    _________________________________________________________________
    batch_normalization_17 (Batc (None, 256, 360, 640)     2560      
    _________________________________________________________________
    reshape (Reshape)            (None, 256, 230400)       0         
    _________________________________________________________________
    permute (Permute)            (None, 230400, 256)       0         
    _________________________________________________________________
    activation_18 (Activation)   (None, 230400, 256)       0         
    =================================================================
    Total params: 10,719,104
    Trainable params: 10,707,744
    Non-trainable params: 11,360
    _________________________________________________________________
    2021-09-22 08:06:17.330092: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open WeightsTracknet/model.1: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
    OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 12 and format 'mp4 / MP4 (MPEG-4 Part 14)'
    OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
    Using device cuda
    Detecting the court and the players...
    /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
      return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
    BOXES  [array([452.1675 , 735.3489 , 572.7479 , 954.95905], dtype=float32)]
    BIGGEST  [452. 735. 573. 955.]
    Finished!
    Tracking the ball: 0.0
    2021-09-22 08:20:51.717046: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
    2021-09-22 08:20:51.724503: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2299995000 Hz
    2021-09-22 08:20:52.243946: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
    2021-09-22 08:20:54.014311: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
    2021-09-22 08:20:54.015598: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
    2021-09-22 08:20:54.015950: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at conv_ops_fused_impl.h:698 : Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    Traceback (most recent call last):
      File "predict_video.py", line 155, in <module>
        pr = m.predict(np.array([X]))[0]
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py", line 1727, in predict
        tmp_batch_outputs = self.predict_function(iterator)
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
        result = self._call(*args, **kwds)
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 957, in _call
        filtered_flat_args, self._concrete_stateful_fn.captured_inputs)  # pylint: disable=protected-access
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 1961, in _call_flat
        ctx, args, cancellation_manager=cancellation_manager))
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 596, in call
        ctx=ctx)
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
        inputs, attrs, num_outputs)
    tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    	 [[node model_1/activation/Relu (defined at predict_video.py:155) ]] [Op:__inference_predict_function_1776]
    
    Function call stack:
    predict_function
    

    😳😳😳😳

    bug documentation good first issue 
    opened by jaweii 3
  • Failed to parse NetParameter file: Yolov3/yolov3.weights in function 'readNetFromDarknet'

    Failed to parse NetParameter file: Yolov3/yolov3.weights in function 'readNetFromDarknet'

    I'm try to build the ""!python3 predict_video.py --input_video_path=VideoInput/video_input3.mp4 --output_video_path=VideoOutput/video_output.mp4 --minimap=0 --bounce=0"" on collab , but I don't see this paths "yolo_weights = 'Yolov3/yolov3.weights'" on the repository.

    Can , you help me to regenerate this weights value please.

    documentation duplicate 
    opened by anjaragit 2
  • Can this be done for a padel court?

    Can this be done for a padel court?

    Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

    This isn't really a bug in the code. I noticed it works smoothly for tennis, but less so for tracking padel. Do you think it's going to be on the roadmap to extend this feature into the padel-sport?

    Describe the solution you'd like A clear and concise description of what you want to happen. Ideally, the option to use this codebase out of the box for padel footage. Realistically, even some guidelines on how to get started with this code to extend it for the padel sport would already be an awesome addition

    Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

    Additional context Add any other context or screenshots about the feature request here.

    opened by KristofVDB1 2
  • no version of Torch

    no version of Torch

    Describe the bug Make new conda env with Python=3.8, run pip install -r requirements.txt and get

    ERROR: Could not find a version that satisfies the requirement torch==1.9.0+cu102 (from versions: 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0, 1.9.1, 1.10.0, 1.10.1, 1.10.2, 1.11.0)
    ERROR: No matching distribution found for torch==1.9.0+cu102
    

    Is cu for cuda? In which case can I run this with another CPU only version of Torch?

    question 
    opened by nyck33 2
  • OpenCV(4.5.4-dev) /tmp/pip-req-build-h45n7_hz/opencv/modules/dnn/src/darknet/darknet_io.cpp:933: error

    OpenCV(4.5.4-dev) /tmp/pip-req-build-h45n7_hz/opencv/modules/dnn/src/darknet/darknet_io.cpp:933: error

    2021-10-26 14:43:32.932732: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open WeightsTracknet/model.1: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 12 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v' Traceback (most recent call last): File "predict_video.py", line 87, in net = cv2.dnn.readNet(yolo_weights, yolo_config) cv2.error: OpenCV(4.5.4-dev) /tmp/pip-req-build-h45n7_hz/opencv/modules/dnn/src/darknet/darknet_io.cpp:933: error: (-213:The function/feature is not implemented) Transpose the weights (except for convolutional) is not implemented in function 'ReadDarknetFromWeightsStream'

    duplicate 
    opened by azuryl 2
  • Questions

    Questions

    Hey, I have three questions:

    1. Is there any way to use this on real time video (webcam, external camera etc...) ?
    2. I want to calculate the speed of ball as well, how can I do this ?
    3. Can you run this project using yolov5 instead yolov3, If so, how?

    Edit: Also, when I tried to "pip install -r requirements.txt" on anaconda process is shows errors after the step "Installing build dependencies" how can I solve this problem Error1

    bug question 
    opened by Draugsleip 1
Owner
ArtLabs
ArtLabs
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Facebook Research 296 Dec 29, 2022
AdelaiDepth is an open source toolbox for monocular depth prediction.

AdelaiDepth is an open source toolbox for monocular depth prediction.

Adelaide Intelligent Machines (AIM) Group 743 Jan 1, 2023
PaddleRobotics is an open-source algorithm library for robots based on Paddle, including open-source parts such as human-robot interaction, complex motion control, environment perception, SLAM positioning, and navigation.

简体中文 | English PaddleRobotics paddleRobotics是基于paddle的机器人开源算法库集,包括人机交互、复杂运动控制、环境感知、slam定位导航等开源算法部分。 人机交互 主动多模交互技术TFVT-HRI 主动多模交互技术是通过视觉、语音、触摸传感器等输入机器人

null 185 Dec 26, 2022
ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

Ibai Gorordo 18 Nov 6, 2022
ONNX-PackNet-SfM: Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Ibai Gorordo 14 Dec 9, 2022
Official Repo for Ground-aware Monocular 3D Object Detection for Autonomous Driving

Visual 3D Detection Package: This repo aims to provide flexible and reproducible visual 3D detection on KITTI dataset. We expect scripts starting from

Yuxuan Liu 305 Dec 19, 2022
HandTailor: Towards High-Precision Monocular 3D Hand Recovery

HandTailor This repository is the implementation code and model of the paper "HandTailor: Towards High-Precision Monocular 3D Hand Recovery" (arXiv) G

Lv Jun 113 Jan 6, 2023
Official implementation of Monocular Quasi-Dense 3D Object Tracking

Monocular Quasi-Dense 3D Object Tracking Monocular Quasi-Dense 3D Object Tracking (QD-3DT) is an online framework detects and tracks objects in 3D usi

Visual Intelligence and Systems Group 441 Dec 20, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Official PyTorch code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction. Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe,

stanley 152 Dec 16, 2022
Code for "NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video", CVPR 2021 oral

NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video Project Page | Paper NeuralRecon: Real-Time Coherent 3D Reconstruction from Mon

ZJU3DV 1.4k Dec 30, 2022
Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Mo

Abhinav Kumar 76 Jan 2, 2023
Categorical Depth Distribution Network for Monocular 3D Object Detection

CaDDN CaDDN is a monocular-based 3D object detection method. This repository is based off of [OpenPCDet]. Categorical Depth Distribution Network for M

Toronto Robotics and AI Laboratory 289 Jan 5, 2023
PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"

FIERY This is the PyTorch implementation for inference and training of the future prediction bird's-eye view network as described in: FIERY: Future In

Wayve 406 Dec 24, 2022
Code and datasets for the paper "Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction" (RA-L, 2021)

Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction This is the code for the paper Combining E

Robotics and Perception Group 69 Dec 26, 2022
Self-Supervised Multi-Frame Monocular Scene Flow (CVPR 2021)

Self-Supervised Multi-Frame Monocular Scene Flow 3D visualization of estimated depth and scene flow (overlayed with input image) from temporally conse

Visual Inference Lab @TU Darmstadt 85 Dec 22, 2022
Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Contact and Human Dynamics from Monocular Video This is the official implementation for the ECCV 2020 spotlight paper by Davis Rempe, Leonidas J. Guib

Davis Rempe 207 Jan 5, 2023
Official implementation of the network presented in the paper "M4Depth: A motion-based approach for monocular depth estimation on video sequences"

M4Depth This is the reference TensorFlow implementation for training and testing depth estimation models using the method described in M4Depth: A moti

Michaël Fonder 76 Jan 3, 2023
Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging This repository contains an implementation

Computational Photography Lab @ SFU 1.1k Jan 2, 2023