3D Player Tracking with Multi-View Stream
Project for 3DV 2021 Spring @ ETH Zurich [Report Link]
This repo contains a full pipeline to support 3D position tracking of soccer players, with multi-view calibrated moving/fixed video sequences as inputs.
- In single-camera tracking stage, Tracktor++ is used to get 2D positions.
- In multi-camera tracking stage, 2D positions are projected into 3D positions. Then across-camera association is achieved as an optimization problem with spatial, temporal and visual constraints.
- In the end, visualization in 2D, 3D and a voronoi visualization for sports coaching purpose are provided.
| 3D Tracking | Sports Coaching |
|---|---|
 |
 |
Demo
Check demo scripts as examples
Currently, processed data is under protection due to legal issues.
- Run the demo visualization on the moving cameras
bash script/demo_moving.sh
- Run the demo visualization on the fixed cameras
bash script/demo_fix.sh
Preprocessing
- Split video into image frames
python src/utils/v2img.py --pathIn=data/0125-0135/CAM1/CAM1.mp4 --pathOut=data/0125-0135/CAM1/img --splitnum=1
- Estimate football pitch homography (size 120m * 90m ref)
python src/utils/computeHomo.py --img=data/0125-0135/RIGHT/img/image0000.jpg --out_dir=data/0125-0135/RIGHT/
- Handle moving cameras
python src/utils/mov2static.py --calib_file=data/calibration_results/0125-0135/CAM1/calib.txt --img_dir=data/0125-0135/CAM1/img --output_dir=data/0125-0135/CAM1/img_static
- Convert ground truth/annotation json to text file
python src/utils/json2txt.py --jsonfile=data/0125-0135/0125-0135.json
Single-camera tracking
- Object Detector: frcnn_fpn
Train object detector and generate detection results with this Google Colab notebook. [pretrained model] - Run Tracktor++
Put trainded object detectormodel_epoch_50.modelintosrc/tracking_wo_bnw/output/faster_rcnn_fpn_training_soccer/.
Put data and calibration results intosrc/tracking_wo_bnw/.
cd src/tracking_wo_bnw
python experiments/scripts/test_tracktor.py
- Run ReID(team id) model
python src/team_classification/team_svm.py PATH_TO_TRACKING_RESULT PATH_TO_IMAGES
- Convert tracking results to coordinates on the pitch
Equation to find the intersection of a line with a plane (ref)
python src/calib.py --calib_path=PATH_TO_CALIB --res_path=PATH_TO_TRACKING_RESULT --xymode --reid
# also plot the camera positions for fixed cameras
python src/calib.py --calib_path=PATH_TO_CALIB --res_path=PATH_TO_TRACKING_RESULT --viz
Across-camera association
- Run two-cam tracker
python src/runMCTRacker.py
# add team id constraint
python src/runMCTRacker.py --doreid
- Run multi-cam tracker (e.g. 8 cams)
python src/runTreeMCTracker.py --doreid
Evaluation
- Produce quatitative results (visualize results)
visualize 2d bounding box
# if format
visualize 3d tracking result with ground truth and voronoi diagram
python src/utils/visualize_on_pitch.py --result_file=PATH_TO_TRACKING_RESULT --ground_truth=PATH_TO_GROUND_TRUTH
visualize 3d ground truth on camera frames (reprojection)
python src/utils/visualize_tracab --img_path=PATH_TO_IMAGES --calib_path=PATH_TO_CALIB --gt_path=PATH_TO_TRACAB_GT --output_path=PATH_TO_OUTPUT_VIDEO
- Produce quantitative result
# 2d
python src/motmetrics/apps/eval_motchallenge.py data/0125-0135/ output/tracktor_filtered
# 3d
python src/utils/eval3d.py --pred=output/pitch/EPTS_3_pitch.txt_EPTS_4_pitch.txt.txt --fixcam --gt=data/fixedcam/gt_pitch_550.txt
python src/utils/eval3d.py --fixcam --boxplot
Acknowledgement
We would like to thank the following Github repos or softwares:
Authors
Yuchang Jiang, Tianyu Wu, Ying Jiao, Yelan Tao
253 Dec 18, 2022
29 Nov 13, 2022
32 Dec 21, 2022
4 Oct 20, 2022
2 Sep 5, 2022
2 Nov 22, 2022
677 Dec 25, 2022
10 Aug 24, 2022
335 Jan 9, 2023
41 Nov 23, 2022
121 Dec 30, 2022
6 Nov 25, 2022
88 Dec 12, 2022
25 Dec 26, 2022
79 Dec 30, 2022
228 Dec 25, 2022
6 Sep 22, 2021
1 Nov 24, 2021
11 Nov 30, 2022