OpenDILab Multi-Agent Environment

Overview

Go-Bigger: Multi-Agent Decision Intelligence Environment

PyPI Anaconda-Server Badge Read the Docs Read the Docs unit_test codecov

GoBigger Doc (中文版)

Ongoing

  • 2021.11.13 We are holding a competition —— Go-Bigger: Multi-Agent Decision Intelligence Environment. Come and make your agents in the game!

GoBigger is a simple and efficient agar-like game engine and provides various interfaces for game AI development. The game is similar to Agar, which is a massively multiplayer online action game created by Brazilian developer Matheus Valadares. In GoBigger, players control one or more circular balls in a map. The goal is to gain as much size as possible by eating food balls and other balls smaller than the player's balls while avoiding larger ones which can eat the player's balls. Each player starts with one ball, but players can split a ball into two when it reaches a sufficient size, allowing them to control multiple balls.

Introduction

GoBigger allows users to interact with the multi-agent environment easily within the basic rules. Through the given interface, users can simply get the observation in game and apply their operations for their agents.

Basic Rules

In order to understand the rules in the game, GoBigger provides a few concepts as following:

  • Match: GoBigger will allow serveral agents (4 by default) to join in a match. There are many different units in a match, such as food balls, thorns balls, spore balls and player balls. Each agent should gain more size by eating other balls to get a higher rank when this match ends.
  • Agent: Each agent control a team including serveral players (3 by default). Teamwork is important for a agent to play against other agents.
  • Player: Each player starts with one ball. In order to improve the operability of the game, GoBigger provides serveral operation for a player ball, including split, eject and stop.
  • Ball: GoBigger provides 4 kinds of balls in a match.
    • Food Ball: Food balls are the neutral resources in the game. If a player ball eat a food ball, the food ball’s size will be parsed to the player ball.
    • Thorn Ball: If a player ball eat a thorns ball, the thorns ball’s size will be parsed to the player ball. But at the same time, the player ball will explode and will be splited into several pieces (10 by default).
    • Spore Ball: Spore balls are ejected by the player balls.
    • Player Ball: Player balls are the balls you can control in the game. You can change its moving direction. In addition, it can eat other balls smaller than itself by covering others’ center.

For more details, please refer to what-is-gobigger.

Observation Space

GoBigger also provide a wealth of observable information, and the observation space can be devided into two part. Here is the brief description of the observation space. For more details, please refer to observation-space.

Global State

Global state provides information related to the whole match, such as the map size, the total time and the last time of the match, and the leaderboard within team name and score.

Player State

Player state should be like:

{
    player_name: {
        'feature_layers': list(numpy.ndarray), # features of player
        'rectangle': [left_top_x, left_top_y, right_bottom_x, right_bottom_y], # the vision's position in the map
        'overlap': {
            'food': [[position.x, position.y, radius], ...], 
            'thorns': [[position.x, position.y, radius], ...],
            'spore': [[position.x, position.y, radius], ...],
            'clone': [[[position.x, position.y, radius, player_name, team_name], ...],     
        }, # all balls' info in vision
        'team_name': team_name, # the team which this player belongs to 
    }
}

We define that feature_layers in player_state represents the feature of this player. feature_layers has several channels, and each channel gives the info of food balls, or spore balls, or thorns balls, or player balls in its vision. For example, in a match we have 4 teams and 3 players for each team, then we get feature_layers as a list, and the length of this list should be 15, including 12 player channel, 1 food ball channel , 1 spore ball channel and 1 thorns ball channel.

Since getting feature_layers costs much time, GoBigger also provides player state without feature_layers when you add use_spatial=False in your render. More details here.

Action Space

In fact, a ball can only move, eject, split, and stop in a match, thus the action space simply includes:

  • Moving direction for the player balls.
  • Split: Players can split a ball into two when it reaches a sufficient size.
  • Eject: Player balls can eject spore on your moving direction.
  • Stop: Stop player balls and gather together together.

More details in action-space.

Getting Started

Installation

Prerequisites

We test GoBigger within the following system:

  • Centos 7.6
  • Windows 10
  • MacOS Catalina 10.15

And we recommend that your python version is 3.6.

Get and install GoBigger

You can simply install GoBigger from PyPI with the following command:

pip install gobigger

If you use Anaconda or Miniconda, you can install GoBigger through the following command:

conda install -c opendilab gobigger

You can also install with newest version through GitHub. First get and download the official repository with the following command line.

git clone https://github.com/opendilab/GoBigger.git

Then you can install from source:

# install for use
# Note: use `--user` option to install the related packages in the user own directory(e.g.: ~/.local)
pip install . --user
     
# install for development(if you want to modify GoBigger)
pip install -e . --user

Launch a game environment

After installation, you can launch your game environment easily according the following code:

import random
from gobigger.server import Server
from gobigger.render import EnvRender

server = Server()
render = EnvRender(server.map_width, server.map_height)
server.set_render(render)
server.start()
player_names = server.get_player_names_with_team()
# get [[team1_player1, team1_player2], [team2_player1, team2_player2], ...]
for i in range(10000):
    actions = {player_name: [random.uniform(-1, 1), random.uniform(-1, 1), -1] \
               for team in player_names for player_name in team}
    if not server.step(actions):
        global_state, screen_data_players = server.obs()
    else:
        print('finish game!')
        break
server.close()

We also build a simple env following gym.Env. For more details, you can refer to gobigger_env.py.

Real-time Interaction with game

GoBigger allow users to play game on their personal computer in real-time. Serveral modes are supported for users to explore this game.

Single Player

If you want to play real-time game on your PC on your own, you can launch a game with the following code:

python -m gobigger.bin.play --player-num 1 --vision-type full

In this mode, up arrow & down arrow & left arrow & rigth arrow allows your balls move, Q means eject spore on your moving direction, W means split your balls, and E means stop all your balls and gather them together.

Double Players

If you want to play real-time game on your PC with your friends, you can launch a game with the following code:

python -m gobigger.bin.play --player-num 2 --vision-type full

In this mode, player1 use up arrow & down arrow & left arrow & rigth arrow allows the balls move, [ means eject spore on your moving direction, ] means split your balls, and \ means stop all your balls and gather them together. player2 use W & S & A & D allows the balls move, 1 means eject spore on your moving direction, 2 means split your balls, and 3 means stop all your balls and gather them together.

Single Players with partial vision

If you want to play real-time game on your PC with only partial vision, you can launch a game with the following code:

python -m gobigger.bin.play --player-num 1 --vision-type partial

Your vision depends on all your balls’ positions and their size.

Single Players against bots

If you want to play against a bot, you can launch a game with the following code:

python -m gobigger.bin.play --vs-bot

You can also add more bots in your game. Try to win the game with more bots!

python -m gobigger.bin.play --vs-bot --team-num 4

High-level Operations in GoBigger

Eject towards the center

Surround others by splitting

Eat food balls quickly

Concentrate size

Resources

For more details, please refer to GoBigger Doc (中文版).

License

GoBigger released under the Apache 2.0 license.

Comments
  • pygame.error: Text has zero width

    pygame.error: Text has zero width

    python3.6 -m gobigger.bin.play --player-num 1 --vision-type full pygame 2.0.3 (SDL 2.0.16, Python 3.6.8) Hello from the pygame community. https://www.pygame.org/contribute.html DEBUG:root:{'team_num': 1, 'player_num_per_team': 1, 'map_width': 1000, 'map_height': 1000, 'match_time': 600, 'state_tick_per_second': 20, 'action_tick_per_second': 5, 'collision_detection_type': 'precision', 'save_video': False, 'save_quality': 'high', 'save_path': '', 'manager_settings': {'food_manager': {'num_init': 2000, 'num_min': 2000, 'num_max': 2500, 'refresh_time': 2, 'refresh_num': 30, 'ball_settings': {'radius_min': 2, 'radius_max': 2}}, 'thorns_manager': {'num_init': 15, 'num_min': 15, 'num_max': 20, 'refresh_time': 2, 'refresh_num': 2, 'ball_settings': {'radius_min': 12, 'radius_max': 20, 'vel_max': 100, 'eat_spore_vel_init': 10, 'eat_spore_vel_zero_time': 1}}, 'player_manager': {'ball_settings': {'acc_max': 100, 'vel_max': 25, 'radius_min': 3, 'radius_max': 300, 'radius_init': 3, 'part_num_max': 16, 'on_thorns_part_num': 10, 'on_thorns_part_radius_max': 20, 'split_radius_min': 10, 'eject_radius_min': 10, 'recombine_age': 20, 'split_vel_init': 30, 'split_vel_zero_time': 1, 'stop_zero_time': 1, 'size_decay_rate': 5e-05, 'given_acc_weight': 10}}, 'spore_manager': {'ball_settings': {'radius_min': 3, 'radius_max': 3, 'vel_init': 250, 'vel_zero_time': 0.3, 'spore_radius_init': 20}}}, 'custom_init': {'food': [], 'thorns': [], 'spore': [], 'clone': []}, 'obs_settings': {'with_spatial': True, 'with_speed': False, 'with_all_vision': False}} Traceback (most recent call last): File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/bin/play.py", line 305, in play_control_by_keyboard() File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/bin/play.py", line 71, in play_control_by_keyboard render.fill(server, direction=None, fps=fps_real, last_time=server.last_time) File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/render/realtime_render.py", line 33, in fill player_num_per_team=1) File "/Users/wduo/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/render/env_render.py", line 219, in render_all_balls_colorful txt = font.render('{}'.format(chr(int(ball.owner)%player_num_per_team+65)), True, WHITE) pygame.error: Text has zero width

    opened by wduo 5
  • Why team num >= 7 is not allowed in Real-time Interaction mode?

    Why team num >= 7 is not allowed in Real-time Interaction mode?

    Hey, I set --team-num 7 and it give me the following error:

    Traceback (most recent call last):
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/bin/play.py", line 311, in <module>
        play_control_by_keyboard_vs_bot(team_num=args.team_num)
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/bin/play.py", line 289, in play_control_by_keyboard_vs_bot
        render.fill(server, direction=None, fps=fps_real, last_time=server.last_time)
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/render/realtime_render.py", line 33, in fill
        player_num_per_team=1)
      File "/home/zhou/miniconda3/envs/gobigger/lib/python3.6/site-packages/gobigger/render/env_render.py", line 314, in render_all_balls_colorful
        pygame.draw.circle(screen, PLAYER_COLORS[int(ball.team_name)][0], ball.position, ball.radius)
    IndexError: list index out of range
    

    and it works fine with team num smaller than 7, what's wrong?

    bug 
    opened by SimZhou 4
  • engine update

    engine update

    • [x] 精简的action space,仅保留move, eject, split三个动作
    • [x] 更真实的引擎,使碰撞相关的计算更方便,并可以总结出一套可量化的移动规则
    • [x] 更快的环境step速度,从整体结构上进行优化
    • [x] 新模式:为每个玩家的每个分身球提供单独的动作
    • [x] 文档更新
    opened by mingzhang96 1
  • Is there any related benchmarks?

    Is there any related benchmarks?

    I knew there was a competition on this environment, so I wonder if there are some benchmark results using latest algorithms, or papers experimented on this environment such that comparisons can be made, thanks!

    opened by GEYOUR 1
  • add density in cfg

    add density in cfg

    说明

    config 中添加了 match_ratio 字段。表示比赛setting的配置比例。默认是1.0。这个值将会乘到manager里涉及到球球数量的参数和地图大小上,来保证等比例对场景进行缩放。例如如果这个值设为0.5,则地图宽高会乘以 math.sqrt(0.5) 来保证面积为原来的0.5倍,然后地图内球球的数量会乘以0.5。

    使用

    server = Server(cfg=dict(match_ratio=1.0))
    
    opened by mingzhang96 1
  • add owner for spore

    add owner for spore

    携带孢子球所属玩家信息 我们为每个孢子球赋予了他被哪个玩家吐出的信息。例如,某个孢子球被id为1的玩家吐出,那么这个孢子球会携带一个 owner 属性,并且值为1.

    举个简单的例子,如果你在 obs_settings 中设置了 with_spore_owner=True,那么在你得到的孢子球信息中将会包含 owner 字段。如下所示:

    [position.x, position.y, radius, owner]
    

    当然,如果你同时设置了 with_speed=True,孢子球信息将会变成如下所示:

    [position.x, position.y, radius, vel.x, vel.y, owner]
    
    opened by mingzhang96 1
  • 0.2.0

    0.2.0

    1. Add owner for spore in overlap #37
    2. Add match_ratio in config to control density #38
    3. allow ball to move over border with center 允许球球跨过地图边界,但是球心不能跨过;allow vision over border 玩家的视野可以跨过地图边界,并且始终是正方形,地图外补零表示 #39
    4. engine update #41
    5. add different config #45
    6. add direction for each cloneball #46
    7. add new replayer for .pb files #47
    8. add more config & udpate doc #51
    opened by mingzhang96 1
  • bug fix

    bug fix

    1. In gobigger/balls/clone_ball.py, self.radius + ball.radius may smaller than d.
    2. In gobigger/players/human_player.py, self.get_clone_num() may equal 0.
    opened by mingzhang96 1
  • add cheat for env render

    add cheat for env render

    Get global vision + player's local vision

    In many scenarios, using some cheat information (such as removing the fog of war) can effectively help the algorithm converge. Therefore, on the basis of obtaining the global vision, we have added a mode of obtaining the global vision and the player's local vision at the same time. Get it by specifying cheat=True. Note that in this mode, the setting of with_all_vision will have no effect, because the global vision information will always be returned. For example, assuming there are 2 teams in a game with 1 player in each team, the player_state obtained will be as follows:

    .. code-block::python

    {
        'all': {
            'feature_layers': list(numpy.ndarray),
            'rectangle': None,
            'overlap': {
                'food': [{'position': position, 'radius': radius}, ...],
                'thorns': [{'position': position, 'radius': radius}, ...],
                'spore': [{'position': position, 'radius': radius}, ...],
                'clone': [{'position': position, 'radius': radius, 'player': player_name, 'team': team_name}, ...],
            },
            'team_name': '',
        }
        '0': {
            'feature_layers': list(numpy.ndarray),
            'rectangle': None,
            'overlap': {
                'food': [{'position': position, 'radius': radius}, ...],
                'thorns': [{'position': position, 'radius': radius}, ...],
                'spore': [{'position': position, 'radius': radius}, ...],
                'clone': [{'position': position, 'radius': radius, 'player': player_name, 'team': team_name}, ...],
            },
            'team_name': team_name,
        },
        '1': {
            'feature_layers': list(numpy.ndarray),
            'rectangle': None,
            'overlap': {
                'food': [{'position': position, 'radius': radius}, ...],
                'thorns': [{'position': position, 'radius': radius}, ...],
                'spore': [{'position': position, 'radius': radius}, ...],
                'clone': [{'position': position, 'radius': radius, 'player': player_name, 'team': team_name}, ...],
            },
            'team_name': team_name,
        },
    }
    

    Note that the global view information is placed under the all field, where the team_name is set to empty. The rest of the player information remains the same.

    opened by mingzhang96 1
  • add reload game for server

    add reload game for server

    Now you can reload a game at any frame number!

    Try to use this feature in your config for server:

    save_bin=False, # Whether to save the information of the game
    load_bin=False, # Whether to load the information of a game at the start of the game
    load_bin_path='', # The file path to load the information of a game at the start of the game
    load_bin_frame_num ='all', # can be int (representing the action frame number to load), or 'all' (representing loading all frames)
    
    documentation enhancement 
    opened by mingzhang96 1
Releases(v0.2.0)
Owner
OpenDILab
Open sourced Decision Intelligence (DI)
OpenDILab
This project uses reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can learn to read tape. The project is dedicated to hero in life great Jesse Livermore.

Reinforcement-trading This project uses Reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can

Deepender Singla 1.4k Dec 22, 2022
Trading and Backtesting environment for training reinforcement learning agent or simple rule base algo.

TradingGym TradingGym is a toolkit for training and backtesting the reinforcement learning algorithms. This was inspired by OpenAI Gym and imitated th

Yvictor 1.1k Jan 2, 2023
Train an RL agent to execute natural language instructions in a 3D Environment (PyTorch)

Gated-Attention Architectures for Task-Oriented Language Grounding This is a PyTorch implementation of the AAAI-18 paper: Gated-Attention Architecture

Devendra Chaplot 234 Nov 5, 2022
A multi-entity Transformer for multi-agent spatiotemporal modeling.

baller2vec This is the repository for the paper: Michael A. Alcorn and Anh Nguyen. baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotempor

Michael A. Alcorn 56 Nov 15, 2022
Multi-task Multi-agent Soft Actor Critic for SMAC

Multi-task Multi-agent Soft Actor Critic for SMAC Overview The CARE formulti-task: Multi-Task Reinforcement Learning with Context-based Representation

RuanJingqing 8 Sep 30, 2022
Spatial Intention Maps for Multi-Agent Mobile Manipulation (ICRA 2021)

spatial-intention-maps This code release accompanies the following paper: Spatial Intention Maps for Multi-Agent Mobile Manipulation Jimmy Wu, Xingyua

Jimmy Wu 70 Jan 2, 2023
A Pytorch implementation of the multi agent deep deterministic policy gradients (MADDPG) algorithm

Multi-Agent-Deep-Deterministic-Policy-Gradients A Pytorch implementation of the multi agent deep deterministic policy gradients(MADDPG) algorithm This

Phil Tabor 159 Dec 28, 2022
Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

RIIT Our open-source code for RIIT: Rethinking the Importance of Implementation Tricks in Multi-AgentReinforcement Learning. We implement and standard

null 405 Jan 6, 2023
Official source code to CVPR'20 paper, "When2com: Multi-Agent Perception via Communication Graph Grouping"

When2com: Multi-Agent Perception via Communication Graph Grouping This is the PyTorch implementation of our paper: When2com: Multi-Agent Perception vi

null 34 Nov 9, 2022
Multi Agent Path Finding Algorithms

MATP-solver Simulator collision check path step random initial states or given states Traditional method Seperate A* algorithem Confict-based Search S

null 30 Dec 12, 2022
A parallel framework for population-based multi-agent reinforcement learning.

MALib: A parallel framework for population-based multi-agent reinforcement learning MALib is a parallel framework of population-based learning nested

MARL @ SJTU 348 Jan 8, 2023
A library of multi-agent reinforcement learning components and systems

Mava: a research framework for distributed multi-agent reinforcement learning Table of Contents Overview Getting Started Supported Environments System

InstaDeep Ltd 463 Dec 23, 2022
Learning to Communicate with Deep Multi-Agent Reinforcement Learning in PyTorch

Learning to Communicate with Deep Multi-Agent Reinforcement Learning This is a PyTorch implementation of the original Lua code release. Overview This

Minqi 297 Dec 12, 2022
Pytorch implementations of popular off-policy multi-agent reinforcement learning algorithms, including QMix, VDN, MADDPG, and MATD3.

Off-Policy Multi-Agent Reinforcement Learning (MARL) Algorithms This repository contains implementations of various off-policy multi-agent reinforceme

null 183 Dec 28, 2022
Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Language Emergence in Multi Agent Dialog Code for the Paper Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog Satwik Kottur, José M.

Karan Desai 105 Nov 25, 2022
WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU

WarpDrive is a flexible, lightweight, and easy-to-use open-source reinforcement learning (RL) framework that implements end-to-end multi-agent RL on a single GPU (Graphics Processing Unit).

Salesforce 334 Jan 6, 2023
Code for Emergent Translation in Multi-Agent Communication

Emergent Translation in Multi-Agent Communication PyTorch implementation of the models described in the paper Emergent Translation in Multi-Agent Comm

Facebook Research 75 Jul 15, 2022
Official Implementation of 'UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers' ICLR 2021(spotlight)

UPDeT Official Implementation of UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers (ICLR 2021 spotlight) The

hhhusiyi 96 Dec 22, 2022
COVINS -- A Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping

COVINS -- A Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping Version 1.0 COVINS is an accurate, scalable, and versatile vis

ETHZ V4RL 183 Dec 27, 2022