The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments.


ABR Control

The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments. ABR Control provides API's for the Mujoco, CoppeliaSim (formerly known as VREP), and Pygame simulation environments, and arm configuration files for one, two, and three-joint models, as well as the UR5 and Kinova Jaco 2 arms. Users can also easily extend the package to run with custom arm configurations. ABR Control auto-generates efficient C code for generating the control signals, or uses Mujoco's internal functions to carry out the calculations.

ABR also provides an interface and config available for controlling a real Jaco 2 at the ABR_Jaco2 repository.


The ABR Control library depends on NumPy, SymPy, SciPy, CloudPickle, Cython, SetupTools, Nengo, and Matplotlib. We recommend using Anaconda. Note that installing in a clean environment will require compiling of the dependent libraries, and will take a few minutes.

To install ABR Control, clone this repository and run:

sudo apt-get install g++
sudo apt-get install python-dev
sudo apt-get install libfreetype6-dev
conda activate your_environment
python install
python develop

ABR Control is tested to work on Python 3.6+, Python 2 is not supported.

Optional Installation


If you would like to use the Mujoco API you will need to install a forked version of mujoco-py with hooks for exitting out of simulations with the ESC key. To use the mujoco API, make sure you are in your anaconda environment and run:

git clone
cd mujoco-py
pip install -e .
pip install glfw>=1.8.3
pip install requests


If you would like to use the Pygame API, from your anaconda environment run:

pip install pygame


We support Vrep <=4.0. You will need to download Vrep and follow the installation instructions.


Some of the path planners work through the use of dynamic movement primitives (DMPs). DMPs allow for a stable, generalizable, and easily extensible way of representing complex trajectories. Path planners making use of DMP are appended with 'dmp' in their filename and will require the installation of the pydmps repository. To install, from your Anaconda environment run:

pip install pydmps


The ABR Control repo is comprised of three parts: 1) arms, 2) controllers, and 3) interfaces.

1a) Arms: Using CoppeliaSim, Pygame, or a real arm

All of the required information about an arm model is kept in that arm's config file. To use the ABR Control library with a new arm, the user must provide the transformation matrices (written using SymPy expressions) from the robot's origin reference frame to each link's centre-of-mass (COM) and joints. These are specified sequentially, e.g. origin -> link0 COM, link0 COM -> joint0, joint0 -> link1 COM, etc. The arm config file and any simulation code is kept in a folder named the same as the arm in the abr_control/arms/ directory.

The ABR Control configuration base class uses the SymPy transform matrices to provide functions that will calculate the transforms, Jacobian, Jacobian derivative, inertia matrices, gravity forces, and centripetal and Coriolis effects for each joint and COM. These can be accessed:

from abr_control.arms import jaco2

robot_config = jaco2.Config()
# calculate the following given the arm state at joint_angles
robot_config.Tx('joint3', q=joint_angles)  # the (x, y, z) position of joint3
robot_config.M(q=joint_angles)  # calculate the inertia matrix in joint space
robot_config.J('EE', q=joint_angles)  # the Jacobian of the end-effector

By default, the use_cython parameter is set to True to allow for real-time control by generating optimized Cython code for each of the robot configuration functions. This can take a little bit of time to generate these functions, but they are saved in ~.cache/abr_control/arm_name/saved_functions where they will be loaded from for future runs. Note that a hash is saved for the config, so if any changes are made the functions will be regenerated during the next use. The cython optimization can be turned off on instantiation:

from abr_control.arms import ur5

robot_config = ur5.Config(use_cython=False)

Below are results from running the operational space controller with different controllers with use_cython=True and False.


1b) Arms: Using Mujoco

When using Mujoco the process is a bit different. Mujoco handles the calculation of all the kinematics and dynamics functions, and only requires an xml config be made describing the kinematic chain. The Mujoco API page describes this in detail.

Detailed models can be created by importing 3D modeling stl files and using the mesh object type in the tag. An example of this is the abr_control/arms/jaco2/jaco2.xml. For users building their own models, you may specify the location of the xml with the folder parameter. For more details, please refer to the Mujoco documentation linked above and use the xml files in this repository as examples.

2) Controllers

Controllers make use of the robot configuration files to generate control signals that accomplish a given task (for most controllers this is reaching a target). The ABR Control library provides implementations of several primary controllers, including operational space, generalized coordinates (joint) space, sliding, and floating control.

When using an operational space controller, it is possible to also pass in secondary controllers to operate in the null space of the operational space controller. These secondary controllers can be set up to achieve secondary goals such as avoiding joint limits and obstacles, damping movement, or maintaining a configuration near a specified resting state.

In the path_planners folder there are several path planners that can be used in conjunction with the controllers. There are filters, linear and second order, which can be used to trace a path from the current position to the target without suddenly warping and causing large spikes in generated torque. The inverse kinematics planner takes in a target for the end-effector and returns a joint angle trajectory to follow. An arc path planner is also provided that creates an arcing path which can be useful when the arm has to reach over itself. This can help prevent self-collisions and odd arm configurations.

Each path planner also has the ability to generate a trajectory for end-effector orientation with the path_plannner.generate_orientation_path() function. This uses spherical linear interpolation (SLERP) to generate a set of orientations from a start to a target quaternion. The time profile will match that of the path planner instantiated (ie: a linear path planner will have a linear step in orientation over time, with a constant change in orientation, whereas a second order path planner will have a bell shaped profile with the largest steps occurring during the middle of the movement, with an acceleration and deceleration at the start and end, respectively.) In addition to filters, there is an example path planner using the dynamic movement primitives trajectory generation system.

Finally, there is an implementation of nonlinear adaptive control in the signals folder, as well as examples in Mujoco, PyGame, and CoppeliaSim showing how this class can be used to overcome unexpected forces acting on the arm.

3) Interfaces

For communications to and from the system under control, an interface API is used. The functions available in each class vary depending on the specific system, but must provide connect, disconnect, send_forces and get_feedback methods.

Putting everything together

A control loop using these three files looks like:

from abr_control.arms import jaco2
from abr_control.controllers import OSC
from abr_control.interfaces import CoppeliaSim

robot_config = jaco2.Config()
interface = CoppeliaSim(robot_config)

ctrlr = OSC(robot_config, kp=20,
            # control (x, y, z) out of [x, y, z, alpha, beta, gamma]
            ctrlr_dof=[True, True, True, False, False, False])

target_xyz = [.2, .2, .5]  # in metres
target_orientation = [0, 0, 0]  # Euler angles, relevant when controlled
for ii in range(1000):
    feedback = interface.get_feedback()  # returns a dictionary with q, dq
    u = ctrlr.generate(
        target=np.hstack([target_xyz, target_orientation]))
    interface.send_forces(u)  # send forces and step CoppeliaSim sim forward


NOTE that when using the Mujoco interface it is necessary to instantiate and connect the interface before instantiating the controller. Some parameters only get parsed from the xml once the arm config is linked to the mujoco interface, which happens upon connection.


The ABR Control repo comes with several examples that demonstrate the use of the different interfaces and controllers.

By default all of the PyGame examples run with the three-link MapleSim arm. You can also run the examples using the two-link Python arm by changing the import statement at the top of the example scripts.

To run the CoppeliaSim examples, have the most recent CoppeliaSim version open. By default, the CoppeliaSim examples all run with the UR5 or Jaco2 arm model. To change this, change which arm folder is imported at the top of the example script. The first time you run an example you will be promted to download the arm model. Simply select yes to download the file and the simulation will start once the download completes.

To run the Mujoco examples, you will be promted to download any mesh or texture files, if they are used in the xml config, similarly to the CoppeliaSim arm model. Once the download completes the simulation will start. If you are using the forked Mujoco-Py repository (See Optional Installation section) you can exit the simulation with the ESC key and pause with the spacebar.

  • Fil r.coppelia sim interface

    Fil r.coppelia sim interface

    Moves from VREP to CoppeliaSim, the new version.

    • all examples updated
    • also fixes a small typo in the README
    opened by studywolf 8
  • Moved scaling function from abr_analyze

    Moved scaling function from abr_analyze

    • function for scaling inputs for spherical conversion using the nengolib function moved from abr_analyze.network_utils to dynamics_adaptation
    • moved test for this function as well to avoid rewriting it
    opened by p3jawors 7
  • xyz_offset argument with problems

    xyz_offset argument with problems

    I've added an ur5 arm on a mobile robot. Because of that i need to constantly update its base position. But when I use the OSC.generate and robot_config.Tx with the specific xyz offset. It returns a different value that is supposed to be. For example:

    base = interface.get_xyz("UR5_link0_visible")
    result: Base is:  [-9.712021827697754, 1.791999101638794, 0.16733066737651825]
    start = robot_config.Tx("EE", feedback["q"], x= np.array(base))
    result: start is:  [-0.33954067  1.79179413 -8.71078803]
    opened by quartzolit 7
  • Added orientation control to OSC

    Added orientation control to OSC

    Added the ability to specify target Euler angles for the end-effector to the operational space controller. It now accepts an additional parameter, ctrlr_dof which is a 6D list specifying which dimensions of [x, y, z, alpha, beta, gamma] you would like to control. NOTE: if the arm under control has fewer DOF than the number of DOF specified for control you will get poor behaviour (usually oscillation between several DOF targets).

    Also changed target_pos -> target in all the controllers, and the input target and target_vel for the OSC must now be 6D vectors.

    Cleaned up the examples, reducing code (removing generation calls outside of main control loop) and removing unused tracking code, updated comments, added two more examples for both the PyGame and VREP folders (orientation_control_osc and position_and_orientation_control_osc), renamed reaching_* -> position_control_*.

    Also I removed the commented code from the OSC null space controller and removed use_dJ as an option.

    opened by studywolf 7
  • error occurs when calculation of 'C'

    error occurs when calculation of 'C'

    When I run file '' to test, an error occurs as follows: 'UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 908: invalid continuation byte', which is in file Line 769 in _calc_C. I compare the calculation of J, M, G, it seems that only calculation C is nor right.

    opened by vanusy 6
  • WIP: Mujoco interface through mujoco_py

    WIP: Mujoco interface through mujoco_py

    Basic interface and example script set up

    • need to resolve the forward kinematics calculations (set up differently than VREP sim)
    • ideally reading from ur5.xml file to get the transformation matrices
    opened by studywolf 6
  • How to create manipulator config files

    How to create manipulator config files

    Hello, I have a new robot config information with urdf format file and D-H parameters, I wish to know how to convert it into config files of this project? The variables in is confusing, can you tell me how to calculate them?

    opened by fhln 5
  • Osc adaptation fixup

    Osc adaptation fixup

    depends on the reach_fixup branch

    this is a some tweaking etc done to improve the dynamics adaptation with osc control

    opened by studywolf 4
  • thanks + pointer to other robot control library

    thanks + pointer to other robot control library

    @studywolf I've been a reader of your blog for quite some time and first I wanted to thank you for all the detailed and informative posts you've written. I saw your recent post about the motion control library.

    I wanted to point you to another set of libraries that go together which I've been using and have found to be quite good for similar purposes:

    This is really just an FYI which incorporates other interesting control approaches and also has been integrated with v-rep in various projects. Hopefully this is more useful or interesting than disruptive, but feel free to close. Thanks! :-)

    opened by ahundt 4
  • What is the effect of dt in your expressions?

    What is the effect of dt in your expressions?

    I can see on your examples you set the dt value in order to connect and set time step of the simulation. But when I designedly increase the dt value, higher is the joint torque value (u) and consequently the control fails at higher dt value. However, no step time is used on your equations, besides when you set up the connection to CoppeliaSim.

    My main objective by asking this is to use your repository via ROS, and controls UR5 robot arm at CoppeliaSim using ROS interface. By doing this it is not possible to force a time step, and both (CoppeliaSim and ROS) will run asynchronously.

    Is that a way to control according to dt, or I need a powerful computer to run both as fast as possible, in order to indirectly generate a lower dt value? (Also thinking in real life robot arm control)

    opened by quartzolit 4
  • Ball joint controller

    Ball joint controller

    Adds joint space control of ball joints into the Joint controller, so that it properly handles computing distance and difference between quaternions and generating angular forces to apply. Also added some examples to test it in the Mujoco examples folder.

    Currently requires users passing in a boolean list identifying which joints are ball joints to the Joint controller, e.g., quaternions=[True, True, False]. Should be able to automate this discovery so the user doesn't have to specify.

    Also makes RestingConfig a subclass of Joint controller, and fixes a bug in DynamicsAdaptation where if the user only specified mean or scale neither were applied.

    opened by studywolf 0
  • joint angles passed in to mujoco_config functions are ignored

    joint angles passed in to mujoco_config functions are ignored

    The values are calculated based on the current state of the arm, not the specified joint angles

    opened by studywolf 0
  • example in Pygame does not work example in Pygame does not work

    Arm goes through obstacle

    opened by p3jawors 1
  • example names

    example names

    having examples named "position control" is probably confusing, since we're doing force control of position, not just position control

    opened by studywolf 0
  • PyPI release

    PyPI release

    It would be nice to have a PyPI release for this repo so that it can be pip installed.

    opened by drasmuss 0
Applied Brain Research
Applied Brain Research
FERM: A Framework for Efficient Robotic Manipulation

Framework for Efficient Robotic Manipulation FERM is a framework that enables robots to learn tasks within an hour of real time training.

Ruihan (Philip) Zhao 82 Jun 9, 2021
DIY split-flap display

The goal is to make a low-cost display that's easy to fabricate at home in small/single quantities (e.g. custom materials can be ordered from Ponoko or similar, and other hardware is generally available).

Scott Bezek 1.7k Jun 13, 2021


wwy 250 Jun 3, 2021
Hotplugger: Real USB Port Passthrough for VFIO/QEMU!

Hotplugger: Real USB Port Passthrough for VFIO/QEMU! Welcome to Hotplugger! This app, as the name might tell you, is a combination of some scripts (py

DARKGuy (Alemar) 28 May 12, 2021
LedFx is a network based LED effect controller with support for advanced real-time audio effects

Welcome to LedFx ✨ -Making music come alive! LedFx website: What is LedFx? What LedFx offers is the ability to take audio input, an

null 247 Jun 11, 2021
Hook and simulate global keyboard events on Windows and Linux.

keyboard Take full control of your keyboard with this small Python library. Hook global events, register hotkeys, simulate key presses and much more.

BoppreH 2.4k Jun 13, 2021
A module for cross-platform control of the mouse and keyboard in python that is simple to install and use.

PyUserInput PyUserInput is a group project so we've moved the project over to a group organization: . That

Paul Barton 972 Jun 6, 2021
Raspberry Pi Pico and LoRaWAN from CircuitPython

Raspberry Pi Pico and LoRaWAN from CircuitPython Enable LoRaWAN communications on your Raspberry Pi Pico or any RP2040-based board using CircuitPython

Alasdair Allan 7 May 24, 2021
Hook and simulate global mouse events in pure Python

mouse Take full control of your mouse with this small Python library. Hook global events, register hotkeys, simulate mouse movement and clicks, and mu

BoppreH 472 Jun 11, 2021
DOS-like OS for RP2040 basic microcontroller boards

Micropython DOS-like OS for RP2040 microcontroller boards. Check out the demo video at To start the OS typ

RetiredWizard 5 Jun 9, 2021
Minimal and clean dashboard to visualize some stats of Pi-Hole with an E-Ink display attached to your Raspberry Pi

Clean Dashboard for Pi-Hole Minimal and clean dashboard to visualize some stats of Pi-Hole with an E-Ink display attached to your Raspberry Pi.

Alessio Santoru 82 May 24, 2021
Micro Displays for Raspberry Pi

micro-displays Micro Displays for Raspberry Pi Why? I'm super bored in lockdown. Add a Raspberry Pi 400 and a few tiny displays... The top half of the

ig 288 Jun 3, 2021
Volkswagen ID component for Home Assistant

Volkswagen ID component for Home Assistant This folder contains both a generic Python 3 library for the Volkswagen ID API and a component for Home Ass

null 19 Jun 10, 2021
A circle of LEDs

This repository contains all the design files, production files and example code for a simple circular LED display.

Pim de Groot 13 May 3, 2021