Scripts for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification

Last update: Jan 1, 2023

Overview

About subwAI

subwAI - a project for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification.

For this project, I made use of a supervised machine learning approach. I provided the ground truth data by playing the game and saving images with the corresponding action that was taken during the respective frame (jump, roll, left, right, noop) and in order for the AI to best imitate my playing style I used a convolutional neural network (CNN) with several layers (convolution, average pooling, dense layer, dropout, output), which gave me a good accuracy of 85% for it's predictions. After augmenting the data (mirroring, which resulted in a dataset twice as big) the model seemed to give even more robust results, when letting it play the game. Ultimately the model managed to finish runs of over a minute regularly and it safely handles the usual obstacles seen in the game. Moreover, the AI - with it's unconvential behavior - discovered a game-changing glitch.

More on all this can be seen in my video on YouTube.

Description/Usage

This repository contains everything that is needed for building an AI that plays Subway Surfers. With the provided scripts you can...

build a dataset by playing the game while running py ai.py gather (takes rapid screenshots of the game and saves images in respective folders ['down', 'left', 'noop', 'right', 'up'] in the folder 'images'); press 'q' or 'esc' to quit
train the specified model defined in get_model() on existing dataset running py ai.py train; add load <image_width> to use a preloaded dataset for the respective image_width provided it has been saved before
augment the existing dataset by flipping every image and adjust the label (flipped image in 'left' needs to be changed to 'right') by running py dataset_augmentation.py
have a look at what your trained model is doing under the hood with py image_check.py to see individual predictions for images and change labels when needed (press 'y' to move on to next image; 'n' to delete image; 'w' to move image to 'up'-folder; 'a' to move image to 'left'-folder; 's' to move image to 'down'-folder; 'd' to move image to 'right'-folder)
if order of images is changed run py image_sort.py in order to bring everything in order again
AND MOST IMPORTANTLY run py ai.py play to let the trained model play the game; press 'q' or 'esc' to quit; press 'y' to save a screen recording after the run and 'n' to not save it; add auto as a command line argument to have the program automatically save recordings of runs longer than 40 seconds

Also...

in the folder 'recordings' you can view the saved screen captures and see the predictions for each individual frame as well as the frame rate
in the folder 'models' your trained models are saved; while the Sequential() model (convolutional neural network with layers defined in get_model()) gives the best results you can also try other more simplistic machine learning models such as [KNeighborsClassifier(n_neighbors=5), GaussianNB(), Perceptron()]
visualizations of the CNN-architecture and details regarding layer configurations as well as the accuracy and loss of the model is saved in models\Sequential

Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

44 Dec 12, 2022

The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Cutoff: A Simple Data Augmentation Approach for Natural Language This repository contains source code necessary to reproduce the results presented in

49 Dec 22, 2022

Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

UniSpeech The family of UniSpeech: UniSpeech (ICML 2021): Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR UniSpeech-

282 Jan 9, 2023

Visual Adversarial Imitation Learning using Variational Models (VMAIL)

Visual Adversarial Imitation Learning using Variational Models (VMAIL) This is the official implementation of the NeurIPS 2021 paper. Project website

14 Nov 18, 2022

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

127 Dec 26, 2022

Image Classification - A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

0 Jan 23, 2022

BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work

BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work. For this project, I used the sigmoid function as an activation function along with stochastic gradient descent to adjust the weights and biases.

1 Jan 22, 2022

From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement (CVPR'2020)

Under-exposure introduces a series of visual degradation, i.e. decreased visibility, intensive noise, and biased color, etc. To address these problems, we propose a novel semi-supervised learning approach for low-light image enhancement.

117 Jan 3, 2023

Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Simple-Image-Classification Simple Image Classification Code (PyTorch) Yechan Kim This repository contains: Python3 / Pytorch code for multi-class ima

8 Oct 29, 2022

Comments

No module for skimage when it's installed trough pip and conda

PS C:\Users\joche\Downloads\subwAI-main> py ai.py gather Traceback (most recent call last): File "C:\Users\joche\Downloads\subwAI-main\ai.py", line 1, in from game import Game File "C:\Users\joche\Downloads\subwAI-main\game.py", line 7, in from skimage.metrics import structural_similarity as ssim ModuleNotFoundError: No module named 'skimage'

opened by jfrenl 0
not able to gather info

PS C:\Users\sasis\Desktop\subwAI-main> python ai.py gather Starting the Game Subway Surfers! Traceback (most recent call last): File "C:\Users\sasis\Desktop\subwAI-main\ai.py", line 431, in main() File "C:\Users\sasis\Desktop\subwAI-main\ai.py", line 49, in main gather_training_data() File "C:\Users\sasis\Desktop\subwAI-main\ai.py", line 145, in gather_training_data game = Game() File "C:\Users\sasis\Desktop\subwAI-main\game.py", line 33, in init subprocess.run(PATH_TO_GAME) File "C:\Users\sasis\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 501, in run with Popen(*popenargs, **kwargs) as process: File "C:\Users\sasis\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 947, in init self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\sasis\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 1416, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified

opened by sasismene 0
"No module named "numpy"

Im not sure how im supposed to run the game because i am new to python but i learned a bit from a youtube video and it says there is no module as numpy. I tried to delete the first 2 lines on notepad++ and then run it then it said no module named mss if i could get help with this it would help alot. Do i need to first download the game or not?

opened by Legend273 1

Can I change to another game

Thanks for sharing the source code

I tried virtuanes

GAME ： Super Mario Bros

PATH_ TO_ GAME = "D:/nes/VirtuaNES.exe"

if keyboard.is_ pressed('a')

if keyboard.is_ pressed('d')

if keyboard.is_ pressed('w')

if keyboard.is_ pressed('s')

....

import mss
# Read full screen or process screen

Can you try out a tutorial with virtuanes

opened by monkeycc 2

Scripts for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification

Related tags

Overview

About subwAI

Description/Usage

You might also like...

Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

Visual Adversarial Imitation Learning using Variational Models (VMAIL)

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

Image Classification - A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work

From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement (CVPR'2020)

Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Comments

No module for skimage when it's installed trough pip and conda

not able to gather info

"No module named "numpy"

Can I change to another game

Owner

This project is a loose implementation of paper "Algorithmic Financial Trading with Deep Convolutional Neural Networks: Time Series to Image Conversion Approach"

NFT-Price-Prediction-CNN - Using visual feature extraction, prices of NFTs are predicted via CNN (Alexnet and Resnet) architectures.

paper: Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

This is a model made out of Neural Network specifically a Convolutional Neural Network model

CNN Based Meta-Learning for Noisy Image Classification and Template Matching

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

《A-CNN: Annularly Convolutional Neural Networks on Point Clouds》(2019)

Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Code repo for "RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network" (Machine Learning and the Physical Sciences workshop in NeurIPS 2021).