An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.

Related tags

Finance tensortrade
Overview

TensorTrade: Trade Efficiently with Reinforcement Learning

Build Status Documentation Status Apache License Discord Python 3.6



TensorTrade is still in Beta, meaning it should be used very cautiously if used in production, as it may contain bugs.

TensorTrade is an open source Python framework for building, training, evaluating, and deploying robust trading algorithms using reinforcement learning. The framework focuses on being highly composable and extensible, to allow the system to scale from simple trading strategies on a single CPU, to complex investment strategies run on a distribution of HPC machines.

Under the hood, the framework uses many of the APIs from existing machine learning libraries to maintain high quality data pipelines and learning models. One of the main goals of TensorTrade is to enable fast experimentation with algorithmic trading strategies, by leveraging the existing tools and pipelines provided by numpy, pandas, gym, keras, and tensorflow.

Every piece of the framework is split up into re-usable components, allowing you to take advantage of the general use components built by the community, while keeping your proprietary features private. The aim is to simplify the process of testing and deploying robust trading agents using deep reinforcement learning, to allow you and I to focus on creating profitable strategies.

The goal of this framework is to enable fast experimentation, while maintaining production-quality data pipelines.

Read the documentation.

Guiding principles

Inspired by Keras' guiding principles.

  • User friendliness. TensorTrade is an API designed for human beings, not machines. It puts user experience front and center. TensorTrade follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear and actionable feedback upon user error.

  • Modularity. A trading environment is a conglomeration of fully configurable modules that can be plugged together with as few restrictions as possible. In particular, exchanges, feature pipelines, action schemes, reward schemes, trading agents, and performance reports are all standalone modules that you can combine to create new trading environments.

  • Easy extensibility. New modules are simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making TensorTrade suitable for advanced research and production use.

Getting Started

You can get started testing on Google Colab or your local machine, by viewing our many examples

Installation

TensorTrade requires Python >= 3.7 for all functionality to work as expected.

pip install -r requirements.txt

Docker

To run the commands below, ensure Docker is installed. Visit https://docs.docker.com/install/ for more information.

Run Jupyter Notebooks

To run a jupyter notebook in your browser, execute the following command and visit the http://127.0.0.1:8888/?token=... link printed to the command line.

make run-notebook

Build Documentation

To build the HTML documentation, execute the following command.

make run-docs

Run Test Suite

To run the test suite, execute the following command.

make run-tests

Support

You can ask questions and join the development discussion:

You can also post bug reports and feature requests in GitHub issues. Make sure to read our guidelines first.

Contributors

Contributions are encouraged and welcomed. This project is meant to grow as the community around it grows. Let me know on Discord in the #suggestions channel if there is anything that you would like to see in the future, or if there is anything you feel is missing.

Working on your first Pull Request? You can learn how from this free series How to Contribute to an Open Source Project on GitHub

https://github.com/notadamking/tensortrade/graphs/contributors

Comments
  • Feature Request: Forex exchange implementation using Meta Trader 4

    Feature Request: Forex exchange implementation using Meta Trader 4

    Funding: 1.75 ETH (~$450 USD)

    Description: Create an implementation of Exchange capable of trading any forex pair on the MetaTrader 4 (MT4) platform.

    Requirements:

    • The exchange implementation must be completely interoperable with MT4, including spinning up any necessary ZeroMQ infrastructure.
    • The exchange must fully implement the Exchange class, including the ability to list balance, net worth, asset valuations, trade history, etc.
    • The exchange must be capable of returning observations from OHLCV, indicators, signals, and other data from the MT4 platform.
    • The exchange must be capable of executing live trades on the MT4 platform, with either live or demo accounts.
    • Must include an example with an agent trading using a demo account.
    enhancement funded $$$$ 
    opened by notadamking 32
  • Feature Request: Support observing and trading multiple assets at a time

    Feature Request: Support observing and trading multiple assets at a time

    Funding: 1.5 ETH (~$300 USD)

    Description: Create the necessary components to support observing multiple assets from an exchange, applying a feature pipeline to those assets, and making a trade on any of those assets.

    Requirements:

    • Create the necessary InstrumentExchange components (or update existing components) to support observing/trading multiple assets on both a simulated and live exchange.
    • Ensure the exchanges support feeding their observations through a simple feature pipeline.
    • Create the necessary ActionStrategy component(s) to support trading multiple assets at a time.
    • Create the necessary RewardStrategy component(s) for rewarding multiple concurrent trades.
    • Create the necessary unit tests to confirm the new components are working as expected.

    TensorTrade tutorial article

    enhancement help wanted funded $$$ 
    opened by notadamking 19
  • Feature Request: Create the render method for a TradingEnvironment.

    Feature Request: Create the render method for a TradingEnvironment.

    Funding: 1.5 ETH (~300 USD)

    Description: Create a useful visualization of the TradingEnvironment in the render method.

    Requirements:

    • Implement a useful visualization of the TradingEnvironment and the underlying the learning agent's interactions with the environment's InstrumentExchange.
    • Display the exchange balance, and all exchange observations/agent trades during each episode in a useful manner.
    • Display the total P/L, sharpe ratio, largest drawdown %, and any metrics you see as useful.
    • The rendered visualization should use a performant, well-designed charting application such as Bokeh or Dash

    Use the RLTrader Github and the corresponding Medium article as a reference.

    enhancement help wanted good first issue funded $$$ 
    opened by notadamking 18
  • Testing Request: Create unit tests for components and documentation

    Testing Request: Create unit tests for components and documentation

    Funding amount: 1.5 ETH (~$300)

    Description: Create fully-featured unit tests for each existing component in /tensortrade and the documentation in /docs.

    Requirements:

    • Each component should be tested via pytest in its corresponding /tests/tensortrade/**/test_*.py file.
    • Each use case of each component should be 100% covered via the unit tests.
    • Each function parameter and class argument that affects a component's functionality should be covered by the unit tests.
    • Each documentation feature and page should be tested via pytest in its corresponding /docs/**/test_*.py file.
    • Each test should be passing.
    • Any bugs in components should be fixed during this process.
    • Note: You are not required to implement unimplemented methods that raise a NotImplementedError().

    TensorTrade tutorial article

    enhancement help wanted good first issue funded $$$$ 
    opened by notadamking 18
  • Feature Request: Enable CUDA GPU acceleration in ParallelDQNAgent

    Feature Request: Enable CUDA GPU acceleration in ParallelDQNAgent

    Funding: $250 USD (GitCoin link pending) Feature: Enable CUDA GPU acceleration in ParallelDQNAgent

    As a user I want to use a CUDA GPU device to accelerate the ParallelDQNAgent's calculations. However, this does not currently work due to a bug with CUDA running in multiple processes.

    Scenario: Train a parallel agent w/ a GPU GIVEN I have a ParallelDQNAgent (agent) AND I am running tensortrade on a machine w/ 4+ cores and 1+ GPU WHEN I run agent.train(n_steps=1000, n_envs=4) THEN the agent should train 1000 steps within 4 separate environments AND the agent should share its model between each of the 4 environments AND the model should utilize the GPU for all possible tensorflow operations

    This feature request is written using Gherkin: https://cucumber.io/docs/gherkin/reference/

    For reference, please look at the ParallelDQNAgent trained in: https://github.com/tensortrade-org/tensortrade/blob/master/examples/train_and_evaluate.ipynb

    enhancement funded $$$ 
    opened by notadamking 16
  • Question: Balances remain in locked balances when after trade has been executed

    Question: Balances remain in locked balances when after trade has been executed

    this is something I noticed during training. That balances will remain in locked_balances when buy -> sell is executed.

    Here is a simple test to illustrate the issue.

    def test_runs_with__buy_sell_error():
        df = pd.read_csv("tests/data/input/coinbase_(BTC,ETH)USD_d.csv").tail(100)
        df = df.rename({"Unnamed: 0": "date"}, axis=1)
        df = df.set_index("date")
    
        ex1 = Exchange("coinbase", service=execute_order)(Stream("USD-BTC", list(df['BTC:close'])))
    
        portfolio = Portfolio(USD, [
            Wallet(ex1, 10000 * USD),
            Wallet(ex1, 0 * BTC)
        ])
    
        coinbase_btc = df.loc[:, [name.startswith("BTC") for name in df.columns]]
    
        nodes = []
        with Module("coinbase") as coinbase:
            for name in coinbase_btc.columns:
                nodes += [Stream(name, list(coinbase_btc[name]))]
    
        feed = DataFeed()(coinbase)
    
        action_scheme = DynamicOrders(
            trade_sizes=[1.0],
            trade_type=TradeType.MARKET
        )
        reward_scheme = SimpleProfit()
    
        env = TradingEnvironment(
            portfolio=portfolio,
            action_scheme=action_scheme,
            reward_scheme=reward_scheme,
            feed=feed,
            window_size=5
        )
    
        env.reset()
        holding_action = 1
        # Buy USD/BTC use all available funds
        buy_all_action = 2
        # Sell BTC/USD use all available funds
        sell_all_action = 3
    
        env.step(buy_all_action)
        assert len(env.broker.executed) == 1
    
        env.step(sell_all_action)
        assert len(env.broker.executed) == 2
         # balances will stay in locked_balances and not return in to balances
    
        env.step(buy_all_action)
        # order will get get executed
        # assert len(env.broker.executed) == 3
    

    Prerequisite: This requires a fix to the class DynamicOrders(ActionScheme): [...] price = exchange.quote_price(pair) instead of price = exchange.quote_price(instrument) [...]

    Maybe its a issue that the order service needs to be adjusted or the default order server is not working with the action scheme? However, I would expect to the traded funds to back into the balances when the env shows the trades as executed and not in locked balances

    question 
    opened by MarcusDoubleYou 12
  • TargetStopActionStrategy - Gitcoin Request

    TargetStopActionStrategy - Gitcoin Request

    This is the requested feature from issue 46 (https://github.com/notadamking/tensortrade/issues/46) It is a variation on DiscreteActionStrategy in which there is a profit target and stop-loss for each trade, which is recorded statefully as a part of the class init, in percentages. Based on the selected profit target and stop-loss, which both default to 1%, if a trade goes above the profit target or below the stop loss (or equals either) the program must use that timestep to trade the oldest amount bought.

    enhancement funded 
    opened by LatiinB 12
  • Drop N frames to ensure that indicators are there

    Drop N frames to ensure that indicators are there

    Background: TT got the completely new data feed system to feed data into the environment. It's based on the Node object that encapsulates the way how data can be organized into a pipeline.

    One Node can feed data to another Node which can go to a third Node that will multiply a value by some other values, that come from different Nodes. The environment just calls feed.next() every step to get the next frame and doesn't know how and where the data come from. The interface above is too generic and flexible that allows easy implementing different data sources, transformations, Talib indicators, etc - the interface remains the same.

    For example, I already implemented a Delay node which returns a value with a predefined delay (to get a price from an hour ago, or a day ago) and a TAlib node which calculates indicators based on an input stream.

    One important implication of this system is that there is no pre-transformation - you just select the source, define the pipeline and pass it to the environment - all transformations and aggregation will be executed as the environment calls feed.next().

    But there is a problem: some node transformations might require some amount of data to be collected before it can return a meaningful result. For example, the calculation talib.SMA(close, timeperiod=30) will require 30 elements before it can return the corresponding SMA. Before that, it will return N/A, which is an invalid value for a neural network.

    You can try to ignore it, and for training, it won't likely affect the model too much, but on a live exchange, your model might make incorrect actions based on incorrect SMA.

    So here is my PR: the ability to tell the environment to ignore first N frames before it actually starts sending a data frame to an agent and executing actions.

    opened by Kukunin 11
  • Documentation Request: Document the existing codebase using ReadTheDocs and Sphinx

    Documentation Request: Document the existing codebase using ReadTheDocs and Sphinx

    Funding amount: 2 ETH (~$400)

    Description: Document each component described in the article, as well as each existing feature in the codebase (Each file in the /tensortrade directory).

    • Write a complete description of each component and file in the /tensortrade directory.
    • Include example usage of each use case and its possible solutions.
    • Briefly define and explain any variables or parameters.
    • Documentation must be publishable via Readthedocs.org using the Sphinx framework.
    • Documentation should be brief, yet complete. It should be simple and easy to read.

    Use the TensorTrade tutorial article as a reference for describing components and defining their usage.

    documentation help wanted good first issue funded $$$$ 
    opened by notadamking 11
  • Feature Request: Replace TA-lib features with bukosabino/ta

    Feature Request: Replace TA-lib features with bukosabino/ta

    Funding: 0.25 ETH (~$50 USD)

    Description: Replace TALibIndicator component in /tensortrade/features/indicators with a component called TaIndicator, that uses the bukosabino/ta package rather than TA-lib.

    Requirements:

    • Replace the TALibIndicator with a TAIndicator.
    • Remove the SimpleMovingAverage indicator, and replace its usage in all examples with the corresponding TAIndicator feature.
    • Create the necessary parameters and kwargs to use all indicators in the bukosabino/ta package within a feature pipeline.

    bukosabino/ta GitHub TensorTrade tutorial article

    enhancement help wanted good first issue funded $ 
    opened by notadamking 10
  • Random starting point in the env to avoid overfitting?

    Random starting point in the env to avoid overfitting?

    Hello, I am testing out PPO with BSH action scheme and position-based returns. I am quite happy with the learned models but it seems to be too overfitted to the train data. In fact, the model is worse for unseen test data. What is your recommendation to avoid overfitting? I am thinking about random starting in the ENV but do not know the right way to implement it? Is there anyone working on it? Thanks

    enhancement 
    opened by trungtv 9
  • [potentially a bug] SimpleOrder: comparing BTC unit to USD unit

    [potentially a bug] SimpleOrder: comparing BTC unit to USD unit

    This part of logic is potentially a bug:https://github.com/tensortrade-org/tensortrade/blob/cc6bd19fbb8451c7972a3a08132407f508c4d47b/tensortrade/env/default/actions.py#L261-L266

    For a USD/BTC pair, when the side is sell, quantity is measured in BTC. However, or size < self.min_order_pct * portfolio.net_worth \ is comparing BTC against USD. Is this potentially a bug?

    What is the purpose of this condition?

    Many thanks!

    opened by spacegoing 0
  • 'EntryPoints' object has no attribute 'get' when importing tensortrade.env.default

    'EntryPoints' object has no attribute 'get' when importing tensortrade.env.default

    Hey,

    When running from tensortrade.env.default I get this error:

    'EntryPoints' object has no attribute 'get'

    Currently running python 3.7.14 on Linux.

    Seems this issue is connected to new release of importlib-metadata 5.0.0 and people mitigate this through downgrading to importlib-metadata==4.13.0.

    I can also confirm it worked in my case.

    opened by Zymoo 0
  • [wip] - support for shorting

    [wip] - support for shorting

    I'm opening a PR to get feedback on adding support for shorting.

    I need this feature, and I'd like to share it with the community too.

    Details

    The implementation intentionally keeps the original behavior in place as much as possible - which is no shorting support . Developers have to be specific in the code to enable it. What this PR adds:

    • MarginWallet - a class that should be explicitly used when shorting is needed. Instead of initializing a Wallet instance, developers can initialize a MarginWallet
    • NegativeQuantity - I created this class as the current implementation is pretty strict with throwing errors for negative quantities, and I wanted to leave that functionality untouched.

    TODO:

    • [ ] Add documentation and sample code showing how to setup shorting.

    Related Issues

    • #317
    • #439
    • #327
    opened by ddragosd 5
  • Rewarding PBR on previous transaction

    Rewarding PBR on previous transaction

    I have implemented the basic ray example found on:

    https://www.tensortrade.org/en/latest/tutorials/ray.html

    and made a basic change to the reward function. The idea was to reward the price difference since the last transaction instead of the price difference since the last step. When I run this example multiple times from fresh new environments I seem to get incoherent results.

    Another slight change from the example is I run tune with tf instead of torch

    I am certain somebody here has though/implemented this logic. Anybody see any issue with this?

    Here is my new reward function

    `

    def __init__(self, price: 'Stream'):
        super().__init__()
        self.position = -1
        self.lastWorth = 0
        self.previousLastWorth = 0
        self.lastAction = 0
    
        price = Stream.sensor(price, lambda p: p.value, dtype="float")
        price = price.rename("price")
        self.priceFeed = DataFeed([price])
        self.priceFeed.compile()
    
        position = Stream.sensor(self, lambda rs: rs.position, dtype="float")
        previousLastWorth = Stream.sensor(self, lambda rs: rs.previousLastWorth, dtype="float")
    
        reward = ((price - previousLastWorth) * position).fillna(0).rename("reward")
        self.rewardFeed = DataFeed([reward])
        self.rewardFeed.compile()
    
    def on_action(self, action: int):
        if self.lastWorth == 0:
            self.position = 0
    
        if abs(action - self.lastAction) > 0:
            self.lastAction = action
            self.position = -1 if action == 0 else 1
            self.previousLastWorth = self.lastWorth
            self.lastWorth = self.priceFeed.next()["price"]
        else:
            self.position = 0
    
    
    def get_reward(self, portfolio: 'Portfolio'):
        return self.rewardFeed.next()["reward"]
    
    def reset(self):
        self.position = -1
        self.rewardFeed.reset()
        self.priceFeed.reset()
        self.lastAction = 0
        self.lastWorth = 0
        self.previousLastWorth = 0
    

    `

    And just in case, here are some versions I am using.

    Ubuntu 22.04.1 LTS Python 3.10.4 tensorflow==2.9.1 tensortrade==1.0.3

    Furthermore, I am not certain if this is a bug, as I am quite new to RL and python, however neither of the links recommended for more generic questions seem to work:

    Please ask your question on StackOverflow or on the TensorTrade Discord #help-desk channel instead of opening a GitHub issue.

    opened by JonathanLacoste 11
  • Minor issue with imports

    Minor issue with imports

    from tensortrade.data.cdd import CryptoDataDownload will load agents even if you don't use them because tensortrade/__init__.py have: from . import agents where tensortrade/agents/__init__.py have from .dqn_agent import DQNAgent, DQNTransition etc

    Got it with unchecked debugger breakpoint in tensortrade/agents/dqn_agent.py

    opened by segatrade 0
Releases(v1.0.3)
Owner
TensorTrade is an open source Python framework for building, training, evaluating, and deploying robust trading algorithms using reinforcement learning.
null
An Algorithmic Trading Library for Crypto-Assets in Python

Service Master Develop CI Badge Catalyst is an algorithmic trading library for crypto-assets written in Python. It allows trading strategies to be eas

Enigma 2.4k Jan 5, 2023
Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)

finmarketpy (formerly pythalesians) finmarketpy is a Python based library that enables you to analyze market data and also to backtest trading strateg

Cuemacro 3k Dec 30, 2022
Github.com/CryptoSignal - #1 Quant Trading & Technical Analysis Bot - 2,100 + stars, 580 + forks

CryptoSignal - #1 Quant Trading & Technical Analysis Bot - 2,100 + stars, 580 + forks https://github.com/CryptoSignal/Crypto-Signal Development state:

Github.com/Signal - 2,100 + stars, 580 + forks 4.2k Jan 1, 2023
Python Backtesting library for trading strategies

backtrader Yahoo API Note: [2018-11-16] After some testing it would seem that data downloads can be again relied upon over the web interface (or API v

DRo 9.8k Dec 30, 2022
Python Algorithmic Trading Library

PyAlgoTrade PyAlgoTrade is an event driven algorithmic trading Python library. Although the initial focus was on backtesting, paper trading is now pos

Gabriel Becedillas 3.9k Jan 1, 2023
:mag_right: :chart_with_upwards_trend: :snake: :moneybag: Backtest trading strategies in Python.

Backtesting.py Backtest trading strategies with Python. Project website Documentation the project if you use it. Installation $ pip install backtestin

null 3.1k Dec 31, 2022
rotki is an open source portfolio tracking, analytics, accounting and tax reporting tool that respects your privacy.

rotki is an open source portfolio tracking, analytics, accounting and tax reporting tool that respects your privacy. The mission of rotki is to bring transparency into the crypto and financial sectors through the use of open source.

Rotki 2k Dec 30, 2022
Python sync/async framework for Interactive Brokers API

Introduction The goal of the IB-insync library is to make working with the Trader Workstation API from Interactive Brokers as easy as possible. The ma

Ewald de Wit 2k Dec 30, 2022
Find big moving stocks before they move using machine learning and anomaly detection

Surpriver - Find High Moving Stocks before they Move Find high moving stocks before they move using anomaly detection and machine learning. Surpriver

Tradytics 1.5k Dec 31, 2022
crypto utilities as a way of learning

cryptos Just me developing a pure Python from-scratch zero-dependency implementation of Bitcoin for educational purposes. This includes a lot of the c

Andrej 958 Jan 2, 2023
Common financial risk and performance metrics. Used by zipline and pyfolio.

empyrical Common financial risk metrics. Table of Contents Installation Usage Support Contributing Testing Installation pip install empyrical Usage S

Quantopian, Inc. 1k Dec 26, 2022
Technical Analysis Library using Pandas and Numpy

Technical Analysis Library in Python It is a Technical Analysis library useful to do feature engineering from financial time series datasets (Open, Cl

Darío López Padial 3.4k Jan 2, 2023
Portfolio and risk analytics in Python

pyfolio pyfolio is a Python library for performance and risk analysis of financial portfolios developed by Quantopian Inc. It works well with the Zipl

Quantopian, Inc. 4.8k Jan 8, 2023
personal finance tracker, written in python 3 and using the wxPython GUI toolkit.

personal finance tracker, written in python 3 and using the wxPython GUI toolkit.

wenbin wu 23 Oct 30, 2022
Fourth and final milestone project

Milestone Project 4: Pound Dog Click link to visit "Pound Dog" Aim of the project The aim of this project is to provide access to a website informing

Jamie Wilson 1 Oct 31, 2021
'Personal Finance' is a project where people can manage and track their expenses

Personal Finance by Abhiram Rishi Pratitpati 'Personal Finance' is a project where people can manage and track their expenses. It is hard to keep trac

Abhiram Rishi Prattipati 1 Dec 21, 2021
Trading Gym is an open source project for the development of reinforcement learning algorithms in the context of trading.

Trading Gym Trading Gym is an open-source project for the development of reinforcement learning algorithms in the context of trading. It is currently

Dimitry Foures 535 Nov 15, 2022
ManiSkill-Learn is a framework for training agents on SAPIEN Open-Source Manipulation Skill Challenge (ManiSkill Challenge), a large-scale learning-from-demonstrations benchmark for object manipulation.

ManiSkill-Learn ManiSkill-Learn is a framework for training agents on SAPIEN Open-Source Manipulation Skill Challenge, a large-scale learning-from-dem

Hao Su's Lab, UCSD 48 Dec 30, 2022