Hi, can MAPPO be used for continuous action space? How can I do this?When I change discrete_action under environment.py to False, the following error will appear.

def train(self): train_infos = [] for agent_id in range(self.num_agents): self.trainer[agent_id].prep_training() train_info = self.trainer[agent_id].train(self.buffer[agent_id]) train_infos.append(train_info)
self.buffer[agent_id].after_update()

    return train_infos

why use after_update() after train()?

Are the choices of IPPO hyperparameters the same as MAPPO shown in Table 12? The only difference is the value of "use_centralized_V" (False for IPPO, True for MAPPO), right? Thanks!

Hello! I found that in your new version of the MAPPO paper, you use a concatenation of the default environment global state, as well as all agents’ local observations, as the mixer network input. But why don't you instead concatenate the Feature-Pruned Agent-Specific Global States which is used in MAPPO to build the input of the mixer network? Is this unfair for the comparison?

如果你们让QMIX用8个进程，增大Batch Size和或者每次训练epoch次数，最后加上 TD(lambda)<=0.5 QMIX能把这些算法干趴下 参考我们的简略调参 https://arxiv.org/abs/2102.03479 我真的发现 MARL这个领域 由于调参的问题导致一大堆错误的结论和实验甚至motivation出发就错了，涉及十篇+ CCFA顶会paper 尤其是 AAAI 这个会议的文章连证明是错的都能 accept

# Normal class FixedNormal(torch.distributions.Normal): def log_probs(self, actions): return super().log_prob(actions).sum(-1, keepdim=True) def entrop(self): return super.entropy().sum(-1) def mode(self): return self.mean

Hi @jxwuyi @eugenevinitsky @zoeyuchao @akashvelu

Thanks for your work!

Just a quick question: is turning on or off the use_centralized_V only affects if the input is from local observations or the centralized state? Does it affect the structure of actual value network used? from the code what I can see is no matter use_centralized_V is true or false, the input size is always (num_agents, input_dim), and the output values are with size (num_agents, 1). So the networks are not affected by use_centralized_V. And centralized value outputs will be the same for "num_agents", right?

Look forward to your reply! Thank you!

Best, Xubo

I have run your code:

./train_smac.sh

however, the following error occured:

env is StarCraft2, map is corridor, algo is mappo, exp is mlp, max seed is 1
seed is 1:
Traceback (most recent call last):
  File "train/train_smac.py", line 175, in <module>
    main(sys.argv[1:])
  File "train/train_smac.py", line 82, in main
    "check recurrent policy!")
AssertionError: check recurrent policy!

I changed the argument "algo" in file "tran_smac.sh"

algo="rmappo"

I don't know whether this modification is suitable or not. It did work, but the results were not satisfactory:

Do you have some good parameters for trainnning?

From what I understand, in MAPE, evaluation is "entangled" in the training mode (there is alteration between training and evaluation phase). Is there any way (or a script) that I can evaluate only a trained agent and save gifs/videos etc?

Both in data collecting and evaluation period, when an episode terminated, the model just take the last obs of corresponding env as input. But i think the envs should be reset if reach termination. Or i missed something in the code?

The output of python is listed here:

Traceback (most recent call last): File "train/train_football.py", line 203, in main(sys.argv[1:]) File "train/train_football.py", line 188, in main runner.run() File "/onpolicy/runner/shared/football_runner.py", line 43, in run self.insert(data) File /onpolicy/runner/shared/football_runner.py", line 141, in insert masks=masks TypeError: insert() got an unexpected keyword argument 'rnn_states'

When I tried to run Hanabi experiments, I was shocked by the time consumption. The CPU I use is 12th Gen Intel(R) Core(TM) i9-12900K and GPU is NVIDIA RTX A5000, neither of which is running at full capacity (approx. 25% - 70%). I use the default parameters in train_hanabi_forward.sh：

python train/train_hanabi_forward.py --env_name ${env} --algorithm_name ${algo} --experiment_name ${exp} --hanabi_name ${hanabi} --num_agents ${num_agents} --seed 4 --n_training_threads 128 --n_rollout_threads 1000 --n_eval_rollout_threads 32 --num_mini_batch 1 --episode_length 100 --num_env_steps 10000000000000 --ppo_epoch 15 --gain 0.01 --lr 7e-4 --critic_lr 1e-3 --hidden_size 512 --layer_N 2 --use_eval --use_recurrent_policy --entropy_coef 0.015 

With this setting, the running FPS is about 3000, meaning it takes 900+ hours to reach the 10B timesteps in the text, i.e. over a month. Is this time consumption consistent with your experience? If not, is there any advice that may fix my problem and improve efficiency?

Hi, I have an issue when reproducing the performance of simple_spread in MPE.

The only modifications on your code:

1. use --use_wandb to disable wandb in train_mpe.sh
2. add self.envs.reset() before line 26 in mpe_runner.py