Gym Threat Defense

Overview

Gym Threat Defense

The Threat Defense environment is an OpenAI Gym implementation of the environment defined as the toy example in Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs by Miehling, E., Rasouli, M., & Teneketzis, D. (2015). It constitutes a 29-state/observation, 4-action POMDP defense problem.

The environment

The Threat Defense environment

Above, the Threat Defense environment can be observed. None of the notations or the definitions made in the paper will be explained in the text that follows, but rather the benchmark of the toy example will be stated. If these are desired, follow the link found earlier to the paper of Miehling, E., Rasouli, M., & Teneketzis, D. (2015).

Attributes

Of the 12 attributes that the toy example is built up by, two are leaf attributes (1 and 5) and one is a critical attribute (12). To give the network a more realistic appearance, the 12 attributes are intepreted in the paper as:

  1. Vulnerability in WebDAV on machine 1
  2. User access on machine 1
  3. Heap corruption via SSH on machine 1
  4. Root access on machine 1
  5. Buffer overflow on machine 2
  6. Root access on machine 2
  7. Squid portscan on machine 2
  8. Network topology leakage from machine 2
  9. Buffer overflow on machine 3
  10. Root access on machine 3
  11. Buffer overflow on machine 4
  12. Root access on machine 4

Actions

The defender have access to the two following binary actions:

  • u_1: Block WebDAV service
  • u_2: Disconnect machine 2

Thus we have four countermeasures to apply, i.e U = {none, u_1, u_2, u_1 & u_2}.

Cost Function

The cost function is defined as C(x,u) = C(x) + D(u).

C(x) is the state cost, and is 1 if the state, that is x, is a critical attribute. Otherwise it is 0.

D(u) is the availability cost of a countermeasure u, and is 0 if the countermeasure is none, 1 if it is u_1 or u_2 and 5 if it is both u_1 and u_2.

Parameters

The parameters of the problem are:

# The probabilities of detection:
beta = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.7, 0.6, 0.7, 0.85, 0.95]

# The attack probabilities:
alpha_1, alpha_5 = 0.5

# The spread probabilities:
alpha_(1,2), alpha_(2,3), alpha_(4,9), alpha_(5,6), alpha_(7,8), alpha_(8,9), alpha_(8,11), alpha_(10,11) = 0.8

alpha_(3,4), alpha_(6,7), alpha_(9,10), alpha_(11,12) = 0.9

# The discount factor:
gamma = 0.85

# The initial belief vector
pi_0 = [1,0,...,0]

Dependencies

  • OpenAI Gym
  • Numpy

Installation

cd gym-threat-defense
pip install -e .

Rendering

There are two possible rendering alternatives when running the environment. These are:

  • Render to stdout
  • A visual mode which prints the graph and indicate which nodes the attacker has taken over

To do a visual rendering, pass in 'rgb_array' to the render function.

env.render('rgb_array')

GUI rendering

Otherwise, for an ASCII representation to stdout, pass in 'human'.

env.render('human')

Example of the printing, where we can see that the agent took the block and disconnect action. The attacker has enabled five attributes, i.e. nodes, represented by ones, where the non-enabled attributes are represented by zeros. A node with parentheses is a leaf node, also known as an entry-point, a square bracket is a normal non-leaf node and a double bracketed node is a critical node.

Action: Block WebDAV service and Disconnect machine 2
(1) --> [1] --> [0] --> [0]
		      \--> [0] <-- [0] <-- [1] <-- [1] <-- (1)
			   \--> [0] <---/
				  \--> [0] --> [[0]]

By default the mode is set to printing to stdout.

Example

As an example on how to use the Threat Defense environment, we provide a couple of algorithms that uses both configurations of the environment. Read the README in the examples/ directory for more information on which algorithm works with which.

Template

How to create new environments for Gym

Inspiration

banana-gym

gym-soccer

gym-pomdp

Authors

You might also like...
A
A "gym" style toolkit for building lightweight Neural Architecture Search systems

A "gym" style toolkit for building lightweight Neural Architecture Search systems

Customizable RecSys Simulator for OpenAI Gym
Customizable RecSys Simulator for OpenAI Gym

gym-recsys: Customizable RecSys Simulator for OpenAI Gym Installation | How to use | Examples | Citation This package describes an OpenAI Gym interfac

Reinforcement Learning with Q-Learning Algorithm on gym's frozen lake environment implemented in python

Reinforcement Learning with Q Learning Algorithm Q learning algorithm is trained on the gym's frozen lake environment. Libraries Used gym Numpy tqdm P

Robot Servers and Server Manager software for robo-gym

robo-gym-server-modules Robot Servers and Server Manager software for robo-gym. For info on how to use this package please visit the robo-gym website

Deep Q Learning with OpenAI Gym and Pokemon Showdown

pokemon-deep-learning An openAI gym project for pokemon involving deep q learning. Made by myself, Sam Little, and Layton Webber. This code captures g

Manipulation OpenAI Gym environments to simulate robots at the STARS lab

Manipulator Learning This repository contains a set of manipulation environments that are compatible with OpenAI Gym and simulated in pybullet. In par

An OpenAI Gym environment for Super Mario Bros
An OpenAI Gym environment for Super Mario Bros

gym-super-mario-bros An OpenAI Gym environment for Super Mario Bros. & Super Mario Bros. 2 (Lost Levels) on The Nintendo Entertainment System (NES) us

Uses Open AI Gym environment to create autonomous cryptocurrency bot to trade cryptocurrencies.

Crypto_Bot Uses Open AI Gym environment to create autonomous cryptocurrency bot to trade cryptocurrencies. Steps to get started using the bot: Sign up

AI virtual gym is an AI program which can be used to exercise and can be used to see if we are doing the exercises

AI virtual gym is an AI program which can be used to exercise and can be used to see if we are doing the exercises

Owner
Hampus Ramström
Hampus Ramström
Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Denoised-Smoothing-TF Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow. Denoised Smoothing is

Sayak Paul 19 Dec 11, 2022
A certifiable defense against adversarial examples by training neural networks to be provably robust

DiffAI v3 DiffAI is a system for training neural networks to be provably robust and for proving that they are robust. The system was developed for the

SRI Lab, ETH Zurich 202 Dec 13, 2022
LBK 20 Dec 2, 2022
Dcf-game-infrastructure-public - Contains all the components necessary to run a DC finals (attack-defense CTF) game from OOO

dcf-game-infrastructure All the components necessary to run a game of the OOO DC

Order of the Overflow 46 Sep 13, 2022
The Malware Open-source Threat Intelligence Family dataset contains 3,095 disarmed PE malware samples from 454 families

MOTIF Dataset The Malware Open-source Threat Intelligence Family (MOTIF) dataset contains 3,095 disarmed PE malware samples from 454 families, labeled

Booz Allen Hamilton 112 Dec 13, 2022
Trading Gym is an open source project for the development of reinforcement learning algorithms in the context of trading.

Trading Gym Trading Gym is an open-source project for the development of reinforcement learning algorithms in the context of trading. It is currently

Dimitry Foures 535 Nov 15, 2022
Plug-n-Play Reinforcement Learning in Python with OpenAI Gym and JAX

coax is built on top of JAX, but it doesn't have an explicit dependence on the jax python package. The reason is that your version of jaxlib will depend on your CUDA version.

null 128 Dec 27, 2022
gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks.

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks. It is built on top of the OpenAI Gym toolkit.

Robin Henry 99 Dec 12, 2022
CL-Gym: Full-Featured PyTorch Library for Continual Learning

CL-Gym: Full-Featured PyTorch Library for Continual Learning CL-Gym is a small yet very flexible library for continual learning research and developme

Iman Mirzadeh 36 Dec 25, 2022
Implement A3C for Mujoco gym envs

pytorch-a3c-mujoco Disclaimer: my implementation right now is unstable (you ca refer to the learning curve below), I'm not sure if it's my problems. A

Andrew 70 Dec 12, 2022