Hi Phil,

I have watched your video on Youtube. There's still a question about the critic_loss.backward(retain_graph=True). In your solution, you just turn the torch version from 1.8.1 to 1.4, I think it's a bug in version 1.4 and so that you are running bug-free in version 1.4.

I have checked a lot of information but I still don't know how to solve it. So here I am to turn to you. Here is my Traceback:

File "main.py", line 101, in <module>
    maddpg_agent.learn(memory)
  File "maddpg.py", line 99, in learn
    critic_loss.backward(retain_graph=True)
  File "/usr/local/lib/python3.7/dist-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 8]], which is output 0 of TBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Hello dear, i'm trying to run the code (after your correction on backward) but i'm getting the following error: I also tried with python 3.6 (numpy 1.14.5 , torch 1.10.1, gym 0.10.5), i'm still getting the same error

critic_loss.backward(retain_graph=True)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/torch/_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/torch/autograd/__init__.py", line 154, in backward
    Variable._execution_engine.run_backward(
RuntimeError: Found dtype Float but expected Double

it seems like the error is being raised from the method MADDPG.learn at : critic_loss.backward(retain_graph=True) i checked the variable of "target" variable and has dtype=torch.float64 Any idea ? thanks

self.target_critic.load_state_dict(critic_state_dict) above code seems make target critic network's parameter always be same as the critic network's. So what is the purpose? making the network learn more slowly？ Hope somebody help me!

https://github.com/philtabor/Multi-Agent-Deep-Deterministic-Policy-Gradients/blob/a3c294aa6834f348a7401306dff3e67919c861f5/maddpg.py#L74

Hi Phill,

Could you please help me to understand what's this line is for? critic_value_new[dones[:, 0.0]] = 0.0 Since critic_value_new float variable it cannot be used as array. Should we set just dones[agent_idx] to 0?

Thanks and Regards Viji

https://github.com/philtabor/Multi-Agent-Deep-Deterministic-Policy-Gradients/blob/a3c294aa6834f348a7401306dff3e67919c861f5/maddpg.py#L83

Dear Phill,

First of all plenty of thanks and gratitude for your lessens, I've learned a lot from your lectures. I've noticed a difference in the code at line 83 in MADDPG class while calculating actor-loss. It's running forward propagation of critic network instead of actor network. I believe this is typo, please correct me if I'm wrong.

Thanks and Regards Viji