Is their an example showing how to train a specific option policy? For example, from the agent flow tutorial, how can we setup training for the ExamplePolicy. The problem being that the output of step in the main_loop may have a different observation_spec and action_spec than the policy.

# Stubs for pulling observation and sending action to some external system.
observation_cb = ExampleObservationUpdater()
action_cb = ExampleActionSender()

# Create an environment that forwards the observation and action calls.
env = ProxyEnvironment(observation_cb, action_cb)

# Stub policy that runs the desired agent.
policy = ExamplePolicy(action_cb.action_spec(), "agent")

# Wrap policy into an agent that logs to the terminal.
task = ExampleSubTask(env.observation_spec(), action_cb.action_spec(), 10)
logger = print_logger.PrintLogger()
aggregator = subtask_logger.EpisodeReturnAggregator()
logging_observer = subtask_logger.SubTaskLogger(logger, aggregator)
agent = subtask.SubTaskOption(task, policy, [logging_observer])

reset_op = ExampleScriptedOption(action_cb.action_spec(), "reset", 3)
main_loop = loop_ops.Repeat(5, sequence.Sequence([reset_op, agent]))

# Run the episode.
timestep = env.reset()
while True:
  action = main_loop.step(timestep)
  timestep = env.step(action)

  # Terminate if the environment or main_loop requests it.
  if timestep.last() or (main_loop.pterm(timestep) > np.random.rand()):
    if not timestep.last():
      termination_timestep = timestep._replace(step_type=dm_env.StepType.LAST)
      main_loop.step(termination_timestep)
    break

Hi guys,

I really like this repository (especially the MoMa) approach. The abstraction into MoMa effectors/sensors is really useful to get the hardware into the loop and I've already managed to integrate the Franka Emika robot pretty well. I'm wondering if you have any plans to integrate upstream development (dm_control/MuJoCo, Python 3.10+). Especially pip packages for the latter would be convenient as Python 3.10 is the system interpreter for Ubuntu 22.04 LTS. I haven't built the wheels for Python 3.10 myself yet, but I think it would be rather straightforward. Newer MuJoCo versions have some breaking changes I think that would require a little more leg work. If you don't have any concrete update plans I'd be happy to start looking into it, once I find the time.

Hi!

Big fan of the subpackages in this repo such as dm_robotics-transformations. Would it be possible to upload binaries that support the latest MuJoCo version? I can't use them at the moment because they require MuJoCo 210.

Cheers!

This pull request updates the dependencies to the currently newest versions of dm_control (1.0.9) and the corresponding MuJoCo release (2.3.1.post1). It also allows to build packages for Python 3.10, which is the system interpreter for the current Ubuntu LTS (22.04).

Hello, there is currently an issue when using the IK modules with dm_control==1.0.1 (error below). This is resolved by using dm_control 1.0.5. Could you please update the dependencies?

Traceback (most recent call last): File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/zerorpc/core.py", line 153, in _async_task functor.pattern.process_call(self._context, bufchan, event, functor) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/zerorpc/patterns.py", line 30, in process_call result = functor(*req_event.args) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/zerorpc/decorators.py", line 44, in call return self._functor(*args, **kargs) File "/home/sasha/Desktop/household_robot_dataset/r2d2/franka/robot.py", line 31, in launch_robot self._ik_solver = RobotIKSolver() File "/home/sasha/Desktop/household_robot_dataset/r2d2/real_robot_ik/robot_ik_solver.py", line 13, in init self._physics = mjcf.Physics.from_mjcf_model(self._arm.mjcf_model) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mjcf/physics.py", line 495, in from_mjcf_model return cls.from_xml_string(xml_string=xml_string, assets=assets) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 424, in from_xml_string return cls.from_model(model) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 407, in from_model return cls(data) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 122, in init self._reload_from_data(data) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mjcf/physics.py", line 530, in _reload_from_data super()._reload_from_data(data) File "/home/sasha/anaconda3/envs/poly-backup/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 370, in _reload_from_data self._warnings_before = np.empty_like(self._warnings) File "<array_function internals>", line 180, in empty_like RuntimeError: Caught an unknown exception!

Hi, the dm_robotics installed is unable to find the header files. Any help would be greatly appreciated!

ERROR: ' In file included from /home/rishabh/workspace/dm_robotics/cpp/mujoco/src/mjlib.cc:15: /home/rishabh/workspace/dm_robotics/cpp/mujoco/include/dm_robotics/mujoco/mjlib.h:22:10: fatal error: mjmodel.h: No such file or directory 22 | #include "mjmodel.h" //NOLINT '

Here's my environment variables

' MJLIB_PATH: /home/rishabh/.mujoco/mujoco/lib/libmujoco.so Using cmake command 'cmake' Using python command '/home/rishabh/anaconda3/envs/packing/bin/python' Using tox command '/home/rishabh/anaconda3/envs/packing/bin/python -m tox' Using python version '3.8' '

Thank you for sharing this great codebase!

I have a question regarding the control frame in sawyer.

What is the reason why cartesian 6d control is expressed in the world's orientation? https://github.com/deepmind/dm_robotics/blob/667a0776fbbe217867fc65d31a3de6749bbf3d54/py/moma/effectors/cartesian_6d_velocity_effector.py#L380

Is there a simple way to modify it to express velocity targets in the local orientation frame?

This pull request is just to improve the wording. The use of "manipulate" was misleading in this context. Replaced with different wording according to the context.

This is my current codebase. I add it together with the other initializers

import dataclasses

from dm_control import mjcf

from rgb_stacking.utils.dr.noise import NoiseDistribution, Normal, LogUniform, Uniform import numpy as np from dm_robotics.moma.entity_initializer import base_initializer

@dataclasses.dataclass class DomainRandomizer(base_initializer.Initializer): def init(self, basket, props, robot): self.props = props self.basket = basket self.arm = robot.arm self.gripper = robot.gripper

    friction, mass, low, hi = np.array([0.9, 0.001, 0.001], float), 0.201, 0.9, 1.1
    self.object_rand = dict(friction=Uniform(friction * low, friction * hi),
                            mass=Uniform(mass * low, mass * hi))

    friction = np.array([0.1, 0.1, 0.0001], float)
    self.arm_rand = dict(friction=Uniform(friction * low, friction * hi),
                         damping=Uniform(0.1 * low, 0.1 * hi),
                         armature=Uniform(low, hi),
                         friction_loss=Uniform(0.3 * low, 0.3 * hi))

    friction = np.array([1, 0.005, 0.0001], float)
    self.hand_rand = dict(friction=Uniform(friction * low, friction * hi),
                          driver_damping=Uniform(0.1 * low, 0.1 * hi),
                          armature=Uniform(low, hi),
                          spring_link_damping=Uniform(0.3 * low, 0.3 * hi))

    friction = np.array([1.0, 0.001, 0.001], float)
    self.basket_friction = Uniform(friction * low, friction * hi)

    friction = np.array([1, 0, 0, 0, 0, 0], float)
    self.actuator_gear = Uniform(friction * low, friction * hi)

def __call__(self, physics: mjcf.Physics, random_state: np.random.RandomState) -> bool:
    for p in self.props:
        collision_geom = p.mjcf_model.find_all('geom')[1]
        collision_geom.friction = self.object_rand['friction'].sample()

    basket_geoms = self.basket.mjcf_model.find_all('geom')
    for b in basket_geoms:
        b.friction = self.basket_friction.sample()

    hand_driver = self.gripper.mjcf_model.find('default', 'driver')
    hand_spring_link = self.gripper.mjcf_model.find('default', 'spring_link')
    hand = self.gripper.mjcf_model.find('default', 'reinforced_fingertip')

    hand.geom.friction = self.hand_rand['friction'].sample()
    hand_driver.joint.armature = self.hand_rand['armature'].sample()
    hand_driver.joint.damping = self.hand_rand['driver_damping'].sample()
    hand_spring_link.joint.damping = self.hand_rand['spring_link_damping'].sample()

    for joint in self.arm.joints:
        joint.armature = self.arm_rand['armature'].sample()
        joint.damping = self.arm_rand['damping'].sample()
        joint.frictionloss = self.arm_rand['friction_loss'].sample()

    for actuator in self.arm.actuators:
        actuator.gear = self.actuator_gear.sample()

    geoms = self.arm.mjcf_model.find_all('geom')
    for g in geoms:
        g.friction = self.arm_rand['friction'].sample()

    return True

Hi,

I keep getting this weird error when calling env.reset(). I only get this error when running a Jupyter notebook

~/miniconda3/lib/python3.9/site-packages/dm_robotics/agentflow/spec_utils.py in debugging_flag() 45 46 def debugging_flag() -> bool: ---> 47 return FLAGS.debug_specs 48 49

Error Message: UnparsedFlagAccessError: Trying to access flag --debug_specs before flags were parsed.

Defining the absl flag by myself does not help: DuplicateFlagError: The flag 'debug_specs' is defined twice. First from dm_robotics.agentflow.spec_utils, Second from /home/ztan/miniconda3/lib/python3.9/site-packages/ipykernel_launcher.py. Description from first occurrence: Debugging switch for checking values match specs.

Any ideas why this keeps happening? Thanks a lot

This line tries to call the update method of dm_control.viewer.gui.glfw_gui.GlfwWindow which doesn't exist. https://github.com/deepmind/dm_robotics/blob/48d5f0bfd76ad497faabd5823a25d89d4526b92e/py/moma/utils/mujoco_rendering.py#L146 The method would be easy to implement (a non-blocking function to render the given pixels similar to the event loop). Interestingly this is not caught by the unit test as that test uses a mock object for the viewer.

Hi, I got this message when I attempt to install dm-robotics-moma/dm-robotics-manipulation.

ERROR: Could not find a version that satisfies the requirement dm-robotics-controllers (from dm-robotics-moma) (from versions: none) ERROR: No matching distribution found for dm-robotics-controllers

I think the issue comes from the fact that dm-robotics-controllers are not fully in python mode. If I drag the whole dm_robotics package to my local directory, where I also extract the controllers folder and put it in the dm_robotics folder, it still doesn't run since files such as cartesian_6d_to_joint_velocity_mapper are written in C++. Is it possible to have a complete python version of the dm_robotics package as soon as possible since I'm working on a robotic project and the deadline is really close. Thanks for your understanding!

FYI I use macOS.