Code for Coordinated Policy Optimization
Webpage | Code | Paper | Talk (English) | Talk (Chinese)
Hi there! This is the source code of the paper “Learning to Simulate Self-driven Particles System with Coordinated Policy Optimization”.
Please following the tutorial below to kickoff the reproduction of our results.
Installation
# Create virtual environment
conda create -n copo python=3.7
conda activate copo
# Install dependency
pip install metadrive-simulator==0.2.3
pip install torch # Make sure your torch is successfully installed! Especially when using GPU!
# Install environment and algorithm.
cd code
pip install -e .
Training
As a quick start, you can start training CoPO in Intersection environment immediately after installation by running:
cd code/copo/
python inter/train_copo_dist.py --exp-name inter_copo_dist
The general way to run training is following:
cd code/copo/
python ENV/train_ALGO.py --exp-name EXPNAME
Here ENV refers to the shorthand of environments:
round # Roundabout
inter # Intersection
bottle # Bottleneck
parking # Parking Lot
tollgate # Tollgate
and ALGO is the shorthand for algorithms:
ippo # Individual Policy Optimization
ccppo # Mean Field Policy Optimization
cl # Curriculum Learning
copo_dist # Coordinated Policy Optimiztion (Ours)
copo_dist_cc # Coordinated Policy Optimiztion with Centralized Critics
finally the EXPNAME is arbitrary name to denote the experiment (with multiple concurrent trials), such as roundabout_copo.
Visualization
We provide the trained models for all algorithms in all environments. A simple command can bring you the visualization of the behaviors of the populations!
cd copo
python vis.py
# In default, we provide you the CoPO population in Intersection environment.
# If you want to see others, try:
python vis.py --env round --algo ippo
# Or you can use the native renderer for 3D rendering:
# (Press H to show helper message)
python vis.py --env tollgate --algo cl --use_native_render
We hope you enjoy the interesting behaviors learned in this work! Please feel free to contact us if you have any questions, thanks!
Citation
@misc{peng2021learning,
title={Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization},
author={Zhenghao Peng and Quanyi Li and Ka Ming Hui and Chunxiao Liu and Bolei Zhou},
year={2021},
eprint={2110.13827},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
My path points to checkpoint-480 as shown. What is the cause of the error? Is it the wrong way to run the script?
But declare checkpoint_name is like: {ALGO}_ {ENV} _{INDEX}.npz in 
Which way should I follow? Do I need to convert checkpoint-xxx files to npz files? How to convert it?
Can the visualization program be started on a remote server?
11 Nov 27, 2022
118 Dec 25, 2022
17 Jan 1, 2023
15 Nov 21, 2022
12 Dec 7, 2022
9 Nov 3, 2021
136 Dec 21, 2022
51 Dec 3, 2022
76 Jan 3, 2023
77 Dec 16, 2022
39 Nov 10, 2022
46 Nov 9, 2022
33 Oct 12, 2022
71 Nov 18, 2022
449 Dec 27, 2022
71 Dec 14, 2022
5 Dec 12, 2022
4 Dec 26, 2021
53 Dec 25, 2022