Use deep learning, genetic programming and other methods to predict stock and market movements

Overview

StockPredictions

Use classic tricks, neural networks, deep learning, genetic programming and other methods to predict stock and market movements.

Both successful and unsuccessful experiments will be posted. This section is things that are currently being explored. Completed projects will be wrapped up and moved to another repository to keep things simple.

The main goal of this project is to learn more about time series analysis and prediction. The stock market just happens to have lots of complicated time series and available data

The first evolving neural net does the best job of predicting daily changes. It's impressive. That'll be my first go to tool

The NASDAQ Evolved Network is a good simple example that should be easy to apply to any index

Data sources:

http://finance.yahoo.com/

https://fred.stlouisfed.org/

https://stooq.com

Data and the cleaning programs:

https://github.com/timestocome/StockMarketData

Recommended Reading:

http://www.e-m-h.org/Fama70.pdf Efficient Market Hypothesis

http://faculty.chicagobooth.edu/workshops/finance/pdf/Shleiferbff.pdf Bubbles for FAMA

http://www.unofficialgoogledatascience.com/2017/04/our-quest-for-robust-time-series.html How Google does series predictions

http://www.econ.ucla.edu/workingpapers/wp239.pdf Let's Take the Con Out of Economics

https://www.manning.com/books/machine-learning-with-tensorflow Meap Machine Learning with TensorFlow

https://www.amazon.com/gp/product/B01AFXZ2F4/ Everybody Lies, Big Data, New Data, and What the Internet can tell us about who we really are

https://www.amazon.com/gp/product/B06XDWV2Z2 The Money Formula: Dodgy Finance, Pseudo Science, and How Mathematicians Took Over the Markets

https://blog.twitter.com/2015/introducing-practical-and-robust-anomaly-detection-in-a-time-series Finding anomalies in time series

https://www.wired.com/2009/02/wp-quant/ Wired: The Formula that Killed Wall St

http://onlinelibrary.wiley.com/doi/10.1111/j.1467-6419.2007.00519.x/abstract What do we know about the profitability of technical analysis

https://eng.uber.com/neural-networks/ Engineering extreme event forecasting at Uber with RNNs

http://lib.ugent.be/fulltxt/RUG01/001/315/567/RUG01-001315567_2010_0001_AC.pdf An empirical analysis of algorithmic trading on financial markets

http://www.radio.goldseek.com/bachelier-thesis-theory-of-speculation-en.pdf The Theory of Speculation, L. Bachelier

http://dl.acm.org/citation.cfm?id=1541882 Anomaly Detection: A Survey 2009 ACM

http://www.mrao.cam.ac.uk/~mph/Technical_Analysis.pdf Technical Analysis

https://is.muni.cz/th/422802/fi_b/bakalarka_final.pdf Prediction of Financial Markets Using Deep Learning ( see: https://github.com/timestocome/FullyConnectedForwardFeedNets for an example fully connected deep learning network )

http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015/j.cumming.pdf An Investigation into the Use of Reinforcement Learning Techniques within the Algorithmic Trading Domain

On my reading list:

http://socserv.mcmaster.ca/racine/ECO0301.pdf Nonparametric Econometrics: A Primer

http://natureofcode.com/ The Nature of Code

http://www.penguinrandomhouse.com/books/314049/scale-by-geoffrey-west/9781594205583/ Scale: The universal laws of growth...

https://en.wikipedia.org/wiki/The_Drunkard%27s_Walk The Drunkard's Walk

Useful Websites:

http://www.nber.org/ The National Bureau of Economic Research

https://fred.stlouisfed.org/ FRED, Federal Reserve Bank of St Louis

http://www.zerohedge.com/ ZeroHedge, mostly noise, occasionally something useful appears

Cool tools:

https://facebookincubator.github.io/prophet/docs/quick_start.html Facebook Prophet - Python and R time series prediction library

https://research.google.com/pubs/pub41854.html Inferring causal impact using bayesian structural time series models ( Google has an R package http://google.github.io/CausalImpact/ to go with this paper )

https://gbeced.github.io/pyalgotrade/ Python Algorithmic Trading Library

http://pybrain.org/ PyBrain Machine Learning Library

https://github.com/CodeReclaimers/neat-python Python NEAT Library for evolving neural networks

Podcasts:

http://www.podcastchart.com/podcasts/berkshire-hathaway-2017-annual-shareholders-meeting/episodes/berkshire-hathaway-vice-chairman-charlie-munger-speaks-with-yahoo-finance-editor-in-chief-andy-serwer 2017 Berkshire Hathaway Shareholder's Meeting

Comments
  • Wanna contribute on this project

    Wanna contribute on this project

    Can I contribute on this project by adding some reading list and useful website??? And later also suggest some ways to predict stock with genetic algorithm?

    opened by grrrrreee 4
  • jupyter / ipython notebook  - a new test with PredictMarketWithBayesTheorem

    jupyter / ipython notebook - a new test with PredictMarketWithBayesTheorem

    any plans to use the jupyter notebook? https://jupyter.org/ graphs e.g. pyplot etc are somewhat easier with that, but that it would probably consume significantly more memory and resources

    i did a new test of PredictMarketWithBayesTheorem in my repository PredictMarketWithBayesTheorem.ipynb

    the main things i changed are normally log returns is computed as log(today's price) - log(yesterday's price) = log ( today's price / yesterday's price ). hence this formula is used instead and for the categories i used a histogram to define them, scroll below for the histogram

    this time round there is a difference, what is most interesting is the HL and HG predictions

    opened by ag88 2
  • Reason behind working on this project

    Reason behind working on this project

    I really liked the way you are working on this project, by not just using the Deep Learning models but also adding traditional technical analysis to it. Can I know why you are working on this? An open-Source Consultation, or a Research Project, an assignment, or just out of interest?

    opened by nikhilrayaprolu 1
  • why are there more README commits in this project?

    why are there more README commits in this project?

    Hello @timestocome I have started researching upon stock market prediction and came across your project. Thank You for your work, but I don't understand why you just update the README all the time :P ?

    opened by nikhilrayaprolu 1
  • Maybe there is another way out

    Maybe there is another way out

    LSTM may not be suitable for predicting trend in a short stock market period.

    I am attempting to classify the shape of a stock(last N days OHLCV data, 5 days for instance) into M kinds of classification (7 output classes).

    Such as drop -8% of stock price as a class, drop [-8%, -4%) as another class, etc.

    As you can see, the training accuracy is high, and definitely overfitting: Accuracy for training classes: (7 classes) [ 82.609 84.146 82.596 87.204 84.164 77.381 80. ]

    Where as the predict accuracy is low: Accuracy for predict following 20 days [ nan 0. 20. 57.143 50. 0. nan]

    For now, I am wondering if change the classification model to DBN could produce a more reasonable data outcome. Hope it helps.

    Below is part of the out fragment:

    Epoch 1996/2000 81581/81581 [==============================] - 58s - loss: 0.0157 - acc: 0.9954 Epoch 1997/2000 81581/81581 [==============================] - 58s - loss: 0.0139 - acc: 0.9957 Epoch 1998/2000 1650/81581 [..............................] - ETA: 57s - loss: 0.0173 - acc: 0.9958 81581/81581 [==============================] - 58s - loss: 0.0143 - acc: 0.9954 Epoch 1999/2000 81581/81581 [==============================] - 58s - loss: 0.0153 - acc: 0.9954 Epoch 2000/2000 5050/81581 [>.............................] - ETA: 55s - loss: 0.0176 - acc: 0.9943 81581/81581 [==============================] - 58s - loss: 0.0158 - acc: 0.9951 save LSTM model... ############## validation on test data ############## scaled data mse: 0.130540770636 load LSTM model... ############## validation on train data ############## scaled data mse: 0.0391746699673 ############## validation on valid data ############## scaled data mse: 0.176731004083 ############## validation on lately data ############## scaled data mse: nan

    ---------- AMD ----------

    classification counter: [23, 82, 339, 422, 341, 84, 30] classification possibility: [ 1.741 6.207 25.662 31.945 25.814 6.359 2.271] classification train predict: [ 82.609 84.146 82.596 87.204 84.164 77.381 80. ] classification valid predict: [ nan 0. 20. 57.143 50. 0. nan]


                       close     volume      predict_profit  a_+1_d  p_+1_d      
    

    Date 2017-03-15 13.98 54885200 -2.360515 -1.0 -2.0 2017-03-16 13.65 44129100 -1.172161 -1.0 -2.0 2017-03-17 13.49 218636000 6.745738 2.0 1.0
    2017-03-20 14.40 90863900 -4.027778 -2.0 0.0 2017-03-21 13.82 72191500 2.026049 1.0 1.0 2017-03-22 14.10 61089400 -2.198582 -1.0 -1.0 2017-03-23 13.79 44144100 -0.652647 0.0 0.0 2017-03-24 13.70 49903700 0.000000 0.0 0.0 2017-03-27 13.70 42537800 -0.072993 0.0 2.0 2017-03-28 13.69 37005800 0.146092 0.0 0.0 2017-03-29 13.71 37777200 2.479942 1.0 -1.0
    2017-03-30 14.05 43814100 3.558719 1.0 -1.0
    2017-03-31 14.55 84362600 0.618557 0.0 0.0
    2017-04-03 14.64 48299200 -3.278689 -1.0 1.0
    2017-04-04 14.16 58217200 0.070621 0.0 -2.0 2017-04-05 14.17 58384000 -6.351447 -2.0 2.0 2017-04-06 13.27 139038000 1.883949 1.0 1.0
    2017-04-07 13.52 70297900 -3.106509 -1.0 1.0 2017-04-10 13.10 46924500 0.000000 0.0 1.0
    2017-04-11 13.10 59786900 -2.595420 -1.0 0.0 2017-04-12 12.76 37087100 NaN NaN 0.0

    opened by doncat99 1
Owner
Linda MacPhee-Cobb
Physicist, Computer Scientist Interests: AI, Machine Learning, Signal Processing, Sensors, Robotics, Evolutionary Algorithms and Hardware
Linda MacPhee-Cobb
This project provides a stock market environment using OpenGym with Deep Q-learning and Policy Gradient.

Stock Trading Market OpenAI Gym Environment with Deep Reinforcement Learning using Keras Overview This project provides a general environment for stoc

Kim, Ki Hyun 769 Dec 25, 2022
:boar: :bear: Deep Learning based Python Library for Stock Market Prediction and Modelling

bulbea "Deep Learning based Python Library for Stock Market Prediction and Modelling." Table of Contents Installation Usage Documentation Dependencies

Achilles Rasquinha 1.8k Jan 5, 2023
Technical experimentations to beat the stock market using deep learning :chart_with_upwards_trend:

DeepStock Technical experimentations to beat the stock market using deep learning. Experimentations Deep Learning Stock Prediction with Daily News Hea

Keon 449 Dec 29, 2022
A Deep Reinforcement Learning Framework for Stock Market Trading

DQN-Trading This is a framework based on deep reinforcement learning for stock market trading. This project is the implementation code for the two pap

null 61 Jan 1, 2023
Predict stock movement with Machine Learning and Deep Learning algorithms

Project Overview Stock market movement prediction using LSTM Deep Neural Networks and machine learning algorithms Software and Library Requirements Th

Naz Delam 46 Sep 13, 2022
This project uses reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can learn to read tape. The project is dedicated to hero in life great Jesse Livermore.

Reinforcement-trading This project uses Reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can

Deepender Singla 1.4k Dec 22, 2022
aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Bayesian Methods for Hackers Using Python and PyMC The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chap

Cameron Davidson-Pilon 25.1k Jan 2, 2023
A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Master status: Development status: Package information: TPOT stands for Tree-based Pipeline Optimization Tool. Consider TPOT your Data Science Assista

Epistasis Lab at UPenn 8.9k Dec 30, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
Stock-history-display - something like a easy yearly review for your stock performance

Stock History Display Available on Heroku: https://stock-history-display.herokua

LiaoJJ 1 Jan 7, 2022
A Genetic Programming platform for Python with TensorFlow for wicked-fast CPU and GPU support.

Karoo GP Karoo GP is an evolutionary algorithm, a genetic programming application suite written in Python which supports both symbolic regression and

Kai Staats 149 Jan 9, 2023
Genetic Programming in Python, with a scikit-learn inspired API

Welcome to gplearn! gplearn implements Genetic Programming in Python, with a scikit-learn inspired and compatible API. While Genetic Programming (GP)

Trevor Stephens 1.3k Jan 3, 2023
A strongly-typed genetic programming framework for Python

monkeys "If an army of monkeys were strumming on typewriters they might write all the books in the British Museum." monkeys is a framework designed to

H. Chase Stevens 115 Nov 27, 2022
Attentional Focus Modulates Automatic Finger‑tapping Movements

"Attentional Focus Modulates Automatic Finger‑tapping Movements", in Scientific Reports

Xingxun Jiang 1 Dec 2, 2021
GazeScroller - Using Facial Movements to perform Hands-free Gesture on the system

GazeScroller Using Facial Movements to perform Hands-free Gesture on the system

null 2 Jan 5, 2022
Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for

Abdultawwab Safarji 7 Nov 27, 2022
Introducing neural networks to predict stock prices

IntroNeuralNetworks in Python: A Template Project IntroNeuralNetworks is a project that introduces neural networks and illustrates an example of how o

Vivek Palaniappan 637 Jan 4, 2023
Introducing neural networks to predict stock prices

IntroNeuralNetworks in Python: A Template Project IntroNeuralNetworks is a project that introduces neural networks and illustrates an example of how o

Vivek Palaniappan 637 Jan 4, 2023
Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution

Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution Figure: Example visualization of the method and baseline as a

Oliver Hahn 16 Dec 23, 2022