Deep Reinforcement Learning for Multiplayer Online Battle Arena

Dohyeong Kim

Last update: Dec 18, 2022

Related tags

Deep Learning MOBA_RL

Overview

MOBA_RL

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Prerequisite

Python 3
gym-derk
Tensorflow 2.4.1
Dotaservice of TimZaman
Seed RL of Google
Ubuntu 20.04
RTX 3060 GPU, 16GB RAM is used to run Dota2 environment with rendering
RTX 3080 GPU, 46GB RAM is used to training 16 number of headless Dota2 environment together in my case

Derk Environment

We are going to train small MOBA environment called Derk.

First, move to dr-derks-mutant-battlegrounds folder.

Run below command to run the 50 parallel environemnt. I modified Seel_RL of Google for my MOBA case.

$ python learner_1.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python learner_2.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python run.py -p1 bot -p2 oldbot -n 50

You can check the training progress using Tensorboard log under tboard path of workspace.

Dota2 Environment

Rendering Environment

You first need to install Dota 2 from Steam. After installation, please check there is Dota2 folder under /home/[your account]/.steam/steam/steamapps/common/dota 2 beta'. We are going to run Dota2 from terminal command.

Next, you need to download and install dotaservice. In my case, I should modity the _run_dota function of dotaservice.py like below.

async def _run_dota(self):
  script_path = os.path.join(self.dota_path, self.DOTA_SCRIPT_FILENAME)
  script_path = '/home/kimbring2/.local/share/Steam/ubuntu12_32/steam-runtime/run.sh'

  # TODO(tzaman): all these options should be put in a proto and parsed with gRPC Config.
  args = [
       script_path,
       '/home/kimbring2/.local/share/Steam/steamapps/common/dota 2 beta/game/dota.sh',
       '-botworldstatesocket_threaded',
       '-botworldstatetosocket_frames', '{}'.format(self.ticks_per_observation),
       '-botworldstatetosocket_radiant', '{}'.format(self.PORT_WORLDSTATES[TEAM_RADIANT]),
       '-botworldstatetosocket_dire', '{}'.format(self.PORT_WORLDSTATES[TEAM_DIRE]),
       '-con_logfile', 'scripts/vscripts/bots/{}'.format(self.CONSOLE_LOG_FILENAME),
       '-con_timestamp',
       '-console',
       '-dev',
       '-insecure',
       '-noip',
       '-nowatchdog',  # WatchDog will quit the game if e.g. the lua api takes a few seconds.
       '+clientport', '27006',  # Relates to steam client.
       '+dota_1v1_skip_strategy', '1',
       '+dota_surrender_on_disconnect', '0',
       '+host_timescale', '{}'.format(self.host_timescale),
       '+hostname dotaservice',
       '+sv_cheats', '1',
       '+sv_hibernate_when_empty', '0',
       '+tv_delay', '0',
       '+tv_enable', '1',
       '+tv_title', '{}'.format(self.game_id),
       '+tv_autorecord', '1',
       '+tv_transmitall', '1',  # TODO(tzaman): what does this do exactly?
  ]

Training Environment

You need to build the Docker image of Dotaservice mentioned in README of Docker of the dotaservice.

You can run the Seel RL for Dota2 using below command.

$ ./run_dotaservice.sh 16
$ ./run_impala.sh 16

Addidinally, you can terminate all process using below command.

$ ./stop.sh

Try out deep learning models online on Google Colab

1.5k Dec 27, 2022

Lighting the Darkness in the Deep Learning Era: A Survey, An Online Platform, A New Dataset

Lighting the Darkness in the Deep Learning Era: A Survey, An Online Platform, A New Dataset This repository provides a unified online platform, LoLi-P

457 Jan 3, 2023

Deep and online learning with spiking neural networks in Python

Introduction The brain is the perfect place to look for inspiration to develop more efficient neural networks. One of the main differences with modern

447 Jan 3, 2023

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

This is the Vowpal Wabbit fast online learning code. Why Vowpal Wabbit? Vowpal Wabbit is a machine learning system which pushes the frontier of machin

8.1k Jan 6, 2023

DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos A concise deep reinforcement learning libr

329 Jan 3, 2023

MazeRL is an application oriented Deep Reinforcement Learning (RL) framework

MazeRL is an application oriented Deep Reinforcement Learning (RL) framework, addressing real-world decision problems. Our vision is to cover the complete development life cycle of RL applications ranging from simulation engineering up to agent development, training and deployment.

222 Dec 24, 2022

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

This is the original implementation of our paper, A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (arXiv:1706.1

1.5k Dec 29, 2022

Deep Reinforcement Learning based Trading Agent for Bitcoin

Deep Trading Agent Deep Reinforcement Learning based Trading Agent for Bitcoin using DeepSense Network for Q function approximation. For complete deta

669 Dec 29, 2022

A list of papers regarding generalization in (deep) reinforcement learning

13 Apr 26, 2021

Comments

ValueError: There is already a bots directory
After downloading and decompressing 'dota2_client_5110' into one folder, I put it the under '~/.local/share/Steam/steamapps/common' and change the path in main.py as below,

game_path = os.path.expanduser("~/.local/share/Steam/steamapps/common/dota2_client_5110/game")

and set the path for run.sh in dotaservice.py as below,

args = [ script_path, '~/.local/share/Steam/steamapps/common/dota2_client_5110/game/dota.sh'

Then I have the output in the terminal indicating:

ValueError: There is already a bots directory (/home/cloudshine/.local/share/Steam/steamapps/common/dota2_client_5110/game/dota/scripts/vscripts/bots)! Please remove manually.
opened by lvnpz 6
some problem about undefined symbol:_ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb

when I run the
python learner_1.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/

I meet some problems like this :

tensorflow.python.framework.errors_impl.NotFoundError: /home/public/grpc/python/../grpc_cc.so: undefined symbol: _ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb

opened by jzl20 4
Cannot download dota2_client_5110 from Google Drive

Hi I had a problem when I download dota2 client 5110. It shows 'zipping' forever, then 'failed to download'. I really appreciate if you could provide any advice.

opened by fangyini 1

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Related tags

Overview

MOBA_RL

Prerequisite

Derk Environment

Dota2 Environment

Rendering Environment

Training Environment

You might also like...

Try out deep learning models online on Google Colab

Lighting the Darkness in the Deep Learning Era: A Survey, An Online Platform, A New Dataset

Deep and online learning with spiking neural networks in Python

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

MazeRL is an application oriented Deep Reinforcement Learning (RL) framework

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

Deep Reinforcement Learning based Trading Agent for Bitcoin

A list of papers regarding generalization in (deep) reinforcement learning

Comments

ValueError: There is already a bots directory

some problem about undefined symbol:_ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb

Cannot download dota2_client_5110 from Google Drive

Owner

Dohyeong Kim

Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

Selfplay In MultiPlayer Environments

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

A resource for learning about deep learning techniques from regression to LSTM and Reinforcement Learning using financial data and the fitness functions of algorithmic trading

Deep Learning and Reinforcement Learning Library for Scientists and Engineers 🔥

Learning to Communicate with Deep Multi-Agent Reinforcement Learning in PyTorch

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

Deep Learning and Reinforcement Learning Library for Scientists and Engineers 🔥

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).