This project provides a stock market environment using OpenGym with Deep Q-learning and Policy Gradient.

Related tags

Deep Learning stock_market_reinforcement_learning
Overview

Stock Trading Market OpenAI Gym Environment with Deep Reinforcement Learning using Keras

Overview

This project provides a general environment for stock market trading simulation using OpenAI Gym. Training data is a close price of each day, which is downloaded from Google Finance, but you can apply any data if you want. Also, it contains simple Deep Q-learning and Policy Gradient from Karpathy's post.

In fact, the purpose of this project is not only providing a best RL solution for stock trading, but also building a general open environment for further research.
So, please, manipulate the model architecture and features to get your own better solution.

Requirements

  • Python2.7 or higher
  • Numpy
  • HDF5
  • Keras with Beckend (Theano or/and Tensorflow)
  • OpenAI Gym

Usage

Note that the most sample training data in this repo is Korean stock. You may need to re-download your own training data to fit your purpose.

After meet those requirements in above, you can begin the training both algorithms, Deep Q-learning and Policy Gradient.

Train Deep Q-learning:

$ python market_dqn.py <list filename> [model filename]

Train Policy Gradient:

$ python market_pg.py <list filename> [model filename]

For example, you can do like this:

$ python market_pg.py ./kospi_10.csv pg.h5

Aware that the provided neural network architecture in this repo is too small to learn. So, it may under-fitting if you try to learn every stock data. It just fitted for 10 to 100 stock data for a few years. (I checked!!)
Thus you need to re-design your own architecture and
let me know if you have better one!

Below is training curve for Top-10 KOSPI stock datas for 4 years using Policy Gradient.
Training Curve

To do

  • Test environment to check overfitting.
  • Elaborate the PG's train interface.

Reference

[1] Playing Atari with Deep Reinforcement Learning
[2] Deep Reinforcement Learning: Pong from Pixels
[3] KEras Reinforcement Learning gYM agents, KeRLym
[4] Keras plays catch, a single file Reinforcement Learning example

You might also like...
Stock-history-display - something like a easy yearly review for your stock performance
Stock-history-display - something like a easy yearly review for your stock performance

Stock History Display Available on Heroku: https://stock-history-display.herokua

LiaoJJ 1 Jan 7, 2022
Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.
Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for

Abdultawwab Safarji 7 Nov 27, 2022
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

pytorch-a2c-ppo-acktr Update (April 12th, 2021) PPO is great, but Soft Actor Critic can be better for many continuous control tasks. Please check out

Ilya Kostrikov 3k Jan 9, 2023
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

Ilya Kostrikov 3k Dec 31, 2022
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

H2O H2O is an in-memory platform for distributed, scalable machine learning. H2O uses familiar interfaces like R, Python, Scala, Java, JSON and the Fl

H2O.ai 6.1k Jan 5, 2023
Predict stock movement with Machine Learning and Deep Learning algorithms

Project Overview Stock market movement prediction using LSTM Deep Neural Networks and machine learning algorithms Software and Library Requirements Th

Naz Delam 46 Sep 13, 2022
DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.
DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos A concise deep reinforcement learning libr

null 329 Jan 3, 2023
Implementation of the bachelor's thesis "Real-time stock predictions with deep learning and news scraping".

Real-time stock predictions with deep learning and news scraping This repository contains a partial implementation of my bachelor's thesis "Real-time

David Álvarez de la Torre 0 Feb 9, 2022
Predicting path with preference based on user demonstration using Maximum Entropy Deep Inverse Reinforcement Learning in a continuous environment
Predicting path with preference based on user demonstration using Maximum Entropy Deep Inverse Reinforcement Learning in a continuous environment

Preference-Planning-Deep-IRL Introduction Check my portfolio post Dependencies Gym stable-baselines3 PyTorch Usage Take Demonstration python3 record.

Tianyu Li 9 Oct 26, 2022
Comments
  • how do you calculate the revenue per episode and got the curve mentioned?

    how do you calculate the revenue per episode and got the curve mentioned?

    I have trained the market_pg.py for 1000 episodes. But to evaluate with new data (2015-08---2016-09), the cumulative reward is nearly zero with whichever the stock chosen. So I want to ask how do you calculate the revenue per episode? Does the cumulative reward means the same thing?

    opened by Syqhhn 0
  • what is the meaning of sudden_death

    what is the meaning of sudden_death

    I do't get the idea of how sudden_death work, can someone explain it? thanks.

    if self.sudden_death * len(self.boughts) > self.reward:
    self.done = True
    opened by scotthuang1989 0
  • what's the meaning of sudden_death?

    what's the meaning of sudden_death?

    I do't get the idea of how sudden_death work, can someone explain it? thanks.

    if self.sudden_death * len(self.boughts) > self.reward:
    self.done = True
    opened by scotthuang1989 0
  • OpenAI gum TypeError: object.__init__() takes no parameters

    OpenAI gum TypeError: object.__init__() takes no parameters

    I got this error when I run

    python3 market_pg.py ./kospi_10.csv pg.h5

    Traceback (most recent call last):
  File "market_pg.py", line 147, in <module>
    env = MarketEnv(dir_path = "./data/", target_codes = list(codeMap.keys()), input_codes = [], start_date = "2010-08-25", end_date = "2015-08-25", sudden_death = -1.0)
  File "/home/mona/stock_market_reinforcement_learning-master/market_env.py", line 69, in __init__
    self.action_space = spaces.Discrete(len(self.actions))
  File "/home/mona/gym-master/gym/spaces/discrete.py", line 13, in __init__
    gym.Space.__init__(self, (), np.int64)
TypeError: object.__init__() takes no parameters

    Is it my openAI version problem ?

    opened by MonaTanggg 1
Owner
Kim, Ki Hyun
Machine Learning Research Engineer
Kim, Ki Hyun
GitHub
Technical experimentations to beat the stock market using deep learning :chart_with_upwards_trend:

DeepStock Technical experimentations to beat the stock market using deep learning. Experimentations Deep Learning Stock Prediction with Daily News Hea

Keon 449 Dec 29, 2022
Use deep learning, genetic programming and other methods to predict stock and market movements

StockPredictions Use classic tricks, neural networks, deep learning, genetic programming and other methods to predict stock and market movements. Both

Linda MacPhee-Cobb 386 Jan 3, 2023
Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO)

V-MPO Simple code to demonstrate Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO) in Pyt

Nugroho Dewantoro 9 Jun 6, 2022
:boar: :bear: Deep Learning based Python Library for Stock Market Prediction and Modelling

bulbea "Deep Learning based Python Library for Stock Market Prediction and Modelling." Table of Contents Installation Usage Documentation Dependencies

Achilles Rasquinha 1.8k Jan 5, 2023
A Deep Reinforcement Learning Framework for Stock Market Trading

DQN-Trading This is a framework based on deep reinforcement learning for stock market trading. This project is the implementation code for the two pap

null 61 Jan 1, 2023
PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

This is the original implementation of our paper, A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (arXiv:1706.1

Zhengyao Jiang 1.5k Dec 29, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
A PyTorch implementation of Learning to learn by gradient descent by gradient descent

Intro PyTorch implementation of Learning to learn by gradient descent by gradient descent. Run python main.py TODO Initial implementation Toy data LST

Ilya Kostrikov 300 Dec 11, 2022
A Closer Look at Invalid Action Masking in Policy Gradient Algorithms

A Closer Look at Invalid Action Masking in Policy Gradient Algorithms This repo contains the source code to reproduce the results in the paper A Close

Costa Huang 73 Dec 24, 2022
Trains an agent with stochastic policy gradient ascent to solve the Lunar Lander challenge from OpenAI

Introduction This script trains an agent with stochastic policy gradient ascent to solve the Lunar Lander challenge from OpenAI. In order to run this

Momin Haider 0 Jan 2, 2022