Neural Dynamic Policies for End-to-End Sensorimotor Learning

I run the code in RL,but I find there maybe some error.and I can not get the good result in paper,could you help me? My success rate always is 0.0,and I always get the error is mujoco_py.builder.MujocoException: Got MuJoCo Warning: Nan, Inf or huge value in QPOS at DOF 0. The simulation is unstable. Time = 2321.0500.

Hello, I have some questions while running your NDP algorithm.

I finished environment settings and run code about 8hrs with $ sh run_rl.sh faucet dmp 2 1

however it seems not working as well.

My questions are

1. How much time to train ppo and ndp algorithm for example environment?
2. How can I know or visualize result plot?
3. I am wondering about dmp_train.py code line 45 to line 72

for j in range(num_updates):
        if args.use_linear_lr_decay:
            utils.update_linear_schedule(
                agent.optimizer, j, num_updates,
                agent.optimizer.lr if args.algo == "acktr" else args.lr)
        envs.reset()
        for step in range(args.num_steps):
            if step % args.T == 0:
                with torch.no_grad():
                    values, actions, action_log_probs_list, recurrent_hidden_states_lst = actor_critic.act(
                        rollouts.obs[step], rollouts.recurrent_hidden_states[step],
                        rollouts.masks[step])

                action = actions[step % args.T]
                action_log_probs = action_log_probs_list[step % args.T]
                recurrent_hidden_states = recurrent_hidden_states_lst[0]
                value = values[:, step % args.T].view(-1, 1)

            obs, reward, done, infos = envs.step(action)

            episode_rewards.append(reward[0].item())
            masks = torch.FloatTensor(
                [[0.0] if done_ else [1.0] for done_ in done])
            bad_masks = torch.FloatTensor(
                [[0.0] if 'bad_transition' in info.keys() else [1.0]
                 for info in infos])
            rollouts.insert(obs, recurrent_hidden_states, action,
                            action_log_probs, value, reward, masks, bad_masks)

in this code, you input same action for step Ts to T(s+1)-1 (T = args.T), but i think that for each step, it is right to set action to actions[step%args.N] since dmp actor outputs N steps(N=args.N) actions. could you explain more detail about this part?

Thank you!

Hi, I am doing this project as my final master project, but when I run the command as your guide, the code can not run due to not found a file '40x40-smnist.mat'. Could you help me solve this problem?

Hi, @shikharbahl, I am struggling with a problem in 'main_il.py'. Is this parameter pt='./dmp/data/pretrained_weights.pt your trained model (ndp_cnn) ? If yes, could you upload this file, please?

@shikharbahl Hi, Could you please confirm 'smnist_loader.py'? 1, is this a '.mat' file type? 2, 'for image in data[image_key][0,0][0]'? 3, 'DMP_data = data[dmp_params_key][0, 0][0]'?

Hello, I am learning about your work and want to run your code. When i run the "main_il.py", it costs much time for a epoch, and I just obtain the result of Epoch 0 as follows: I want to know that common time spent on this, and is there any mistake that I made for this? I'd appreciate it if you could help me, thank you.

Hi,

I am a student from Karlsruhe Institute of Technology and I would like to have a deeper look of your work and experiment.

I followed the readme and create a conda env but I got stuck in the pip packages installation. It looks like few pip dependencies are not available any more. Could you please update those dependencies?

I will appreciate any suggestion from yours, thx!

Best, Ge Li