Hello,

Thanks for helping with the last issue about the Humanoid-v2 task. I got another issue while reproducing the RL experiments in the paper. I noticed that you mentioned in the paper that you used some product groups as the approximate symmetry groups such as Z2 x Z2 (Swimmer), D4 x O(2) (Ant), and Z2 x O(2) (Humanoid). However, "RL/representations.py" doesn't seem to include such product groups, while I was able to only find Z2 or Z4.

I want to apply the product groups for the corresponding RL tasks. I wish to know the hidden implementation that I haven't found or any advice to implement such product groups in the RL tasks. Only one example would be enough like Z2 x O(2) (Humanoid).

Hi, I tried the following command to test the Humanoid-v2 task python train_rpp.py --env_name=Humanoid-v2 --save_dir=./tmp/rpp --rpp_value=False However, it didn't work and raised those error messages

Warning: Flow failed to import. Set the environment variable D4RL_SUPPRESS_IMPORT_ERROR=1 to suppress this message.
No module named 'flow'
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/glfw/__init__.py:906: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
  warnings.warn(message, GLFWError)
Warning: CARLA failed to import. Set the environment variable D4RL_SUPPRESS_IMPORT_ERROR=1 to suppress this message.
No module named 'carla'
pybullet build time: Apr 26 2022 03:12:14
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/jax/_src/numpy/lax_numpy.py:5406: UserWarning: 'kind' argument to argsort is ignored; only 'stable' sorts are supported.
  warnings.warn("'kind' argument to argsort is ignored; only 'stable' sorts "
CWD =  /workspace/siml/hyunsu/temp/RL
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/gym/envs/registration.py:505: UserWarning: WARN: The environment Humanoid-v2 is out of date. You should consider upgrading to version `v3` with the environment ID `Humanoid-v3`.
  logger.warn(
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/gym/core.py:172: DeprecationWarning: WARN: Function `env.seed(seed)` is marked as deprecated and will be removed in the future. Please use `env.reset(seed=seed) instead.
  deprecation(
tau =  0.005
rpp policy is True
Traceback (most recent call last):
  File "train_rpp.py", line 187, in <module>
    app.run(main)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "train_rpp.py", line 105, in main
    agent = SACLearner(FLAGS.seed,
  File "/workspace/siml/hyunsu/temp/RL/jax_rl/agents/sac/sac_learner.py", line 126, in __init__
    actor_def = policies.RPPNormalTanhPolicy(state_rep,action_rep,action_std_rep,symmetry_group,ch,
  File "/workspace/siml/hyunsu/temp/RL/jax_rl/networks/policies.py", line 25, in RPPNormalTanhPolicy
    state_rep = state_rep(G)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/product_sum_reps.py", line 69, in __call__
    return SumRepFromCollection({rep(G):c for rep,c in self.reps.items()},perm=self.perm)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/product_sum_reps.py", line 69, in <dictcomp>
    return SumRepFromCollection({rep(G):c for rep,c in self.reps.items()},perm=self.perm)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/representation.py", line 49, in __call__
    raise NotImplementedError
NotImplementedError

The error doesn't look an issue of wrong package versions. Could you help me to resolve it? For reference, the only two things I modified in the code were

1. Converting every "@jax.partial" to "@partial" of functools.
2. Revising the PseudoScalar class from

class PseudoScalar(Rep):
    is_regular=False
    def __init__(self,G=None):
        self.G=G
        self.concrete = (self.G is not None)

to

class PseudoScalar(Rep):
    is_regular=False
    def __init__(self,G=None):
        self.G=G
        # self.concrete = (self.G is not None)
    @property
    def concrete(self):
        return self.G is not None

as in the Rep class, and here are the versions of related packages

• python 3.8
• jax==0.3.1
• jaxlib==0.3.0+cuda11.cudnn805
• flax==0.3.6
• emlp==1.0.3
• gym==0.23.1
• mujoco==2.1.5
• gdown==4.4.0