Python Dash app that tracks whale activity in cryptocurrency markets.

Overview

Introduction

Welcome! This is a Python-based Dash app meant to track whale activity in buy / sell walls on crypto-currency exchanges (presently just operational for GDAX, but more exchanges to come). This document aims to explain the purpose, functionality, and future of this project. Please do share this with your fellow coders / traders / crypto-aficionados, and contribute to the future of this project by calling out issues, requesting new features, and submitting pull requests to improve the app.

The newest version of this app is quite computationally intense, and as such, it is recommended that the new version be run locally pursuant to the instructions below. If this version gives you trouble at all, a less computationally-intense earlier version of the app can be found at this commit version here, and thanks to a generous member of the community, is hosted online here.

If anyone has questions about the app, feel free to reach out to open an issue.

Scroll down to the How-To / User Guide section to learn more about how to run the app yourself. The most recent updates to the app include new features like an improved depth chart (shown in the screenshot below), as well as a side-menu that allows selective hiding / showing of various pairs of interest.

First UI View

What's the point of this app?

Presently, GDAX allows users to see the buy and sell limit order volume via the "depth chart" shown below.

The problem with this is that the depth chart does not tell us where the volume behind buy and sell walls is coming from. It could be from a few whales seeking to manipulate the market, or it could be from a large number of individuals who have placed orders around a modal point. In technical terms, there is no way to see how much of present resistance or support levels are due to individuals vs. group clustering.

This difference is quite important, as it impacts how quickly walls can be pulled, etc., so having buy & sell-wall information at the order level can prove quite valuable for traders. This simple app allows you to access just that information, by focusing on individual limit orders that constitute large walls, and particularly emphasizing the largest orders, enabling the user to spot whales that may be manipulating the present price via various methods. How exactly we define / spot whale activity is the focus of the next section.

How to spot a whale:

This is perhaps the most important question when it comes to the purpose of this app. We have iterated on this many times, and have arrived at an algorithmic definition that spots 2 types of whales:

  • Single Price-Point Whales:
    • Place one large order at a single price-point
    • Example: 500 ETH for sale at $1000 via 1 unique order
    • Represented via a bubble in the visualization.
    • Tooltip includes order price-point, volume, and number of unique orders
  • Ladder Price-Point Whales:
    • Place many medium-sized orders for identical volume increasing in price
    • Example: 500 total ETH for sale starting at $900 and spanning until $1000 via 10 unique 50-ETH orders
    • Represented via a line bar spanning the range of the ladder
    • Tooltip includes number and volume of orders, price-range of ladder, and total volume

While these two may seem different, their market impacts are often comparable. I think a real world analogy helps to illustrate this point:

Imagine that you are looking to understand if there is any whale activity at your local coffee shop (I know this is silly but I promise it is a good example; stay with me here). You could sit near the ordering area and write down each individual order. If someone comes along and orders 100 lattes, that would get your attention--you have spotted a whale! What if, however, someone was ordering 5 lattes, and then getting in the back of the line again, and ordering another 5 lattes, repeating this until they had purchased a total of 100 lattes? Both of these individuals had a whale-like impact on market movement, but if all you care about is the size of each individual order, you will miss out on the sneaky whales that may be splitting their orders across a ladder of price-points.

Given that GDAX's API doesn't display ETH addresses (only unique order IDs), we cannot say with 100% certainty that what looks like a "ladder price-point whale" is in fact a unitary actor. We have to make some assumptions. To continue the coffee shop metaphor, we aren't able to actually look at the people in line for coffee; instead, we have a blindfold on and are jotting down what we hear the barista say. We hear "1 order for 5 lattes" once and write it down, but if we hear "1 order for 5 lattes" 100 different times in close succession, then we can make the safe assumption that we are dealing with a whale, as this is analogous to seeing 100 orders for 10 ETH stacked in a ladder of price-points in close succession on GDAX.

As can be seen from the UI screenshot below, for each currency pairing the user can easily examine the most obvious whales hiding amongst the walls. The algorithm used displays only those orders that make up >= 1% of the volume of the portion of the order book shown in the visualization (which is presently +/-5% from present market price). This methodology makes our analysis robust both in times of price stability--when there is both resistance and support similar in magnitude--as well as during times of large price fluctuation--when there may be very little support or very little resistance. Thanks to the creative coloring algorithm behind the visualization, the brightest colors are those most likely to be whales. The colors become progressively darker as the number of distinct orders at a price-point increases, allowing for easy visual identification of whales in the market.

Main UI

In addition to the main views which provide at-a-glance information about the largest orders, users can freeze the live refresh within the app in order to zoom in on particular sections of the order book, or to take better advantage of the tooltip capabilities of the Plotly visualization. Given that we now track both single price-point and ladder price-point whales, there are often times when bubbles overlap the linebar charts. When this happens, simply freeze the visualization and zoom in on a particular area to separate the two in a more detailed view. An example of the tooltip functionalities for both the ladder and single price-point whales can be seen via the screenshots below.

Single-Price-Point Whale

In the image above we are looking at a zoom-in on the chart of BTC/USD. The bubble that we have hovered over near the top towards the center of the view is a slightly dark red bubble (as opposed to the brightest red) because there are 2 orders at that exact price price-point; hence, we cannot be 100% confident that it is one whale acting unilaterally. That said, those 2 orders alone constitute almost $1,000,000.00 in sell-side pressure, so they are important to call out, and thus our methodology captures them. We can see additional useful information via the tooltip--namely, that these 2 sell orders together constitute 113.38 BTC in sell-order volume, with each of those orders being placed at a price-point of $8,352.00. Lastly, we see the count of unique orders at this price point (2), along with the sum total value of those orders ($946,949.76).

Ladder-Price-Point-Whale

In the image above we are looking at the chart for ETH/USD, hovering over a large ladder price-point whale. Here we can see that the tooltip calls out the total number of rungs on the ladder--7 orders of 50 ETH each--as well as the price range in which those orders are found--from $506.99 to $520.00--and the total market value of those orders (in this case $179,051.50). The thickness of the line is also proportional to the size of each rung on the order ladder.

We cannot be as certain that ladder price-point whales are, in fact, individual whales, as it could just be a few people who place similar orders of identical size at different psychological modal points, but in most cases this is unlikely, and we leave it up to the user to make the ultimate determination as to whether or not they believe a particular order or set of orders to be a whale or not. That said, we do recognize that the ladder price-point whale methodology could perhaps be improved with variable controls, and we are actively looking into this along with other improvements. Our app only serves to provide the information, and to ascribe some semblance of a probability to the likelihood that an observation is a whale or not.

How-To / User Guide (and a few more technical details)

The present version tracks all major pairings (ETH/USD; ETH/BTC; BTC/USD; LTC/USD) but I can add more upon request. It is set to update every 5 seconds (to optimize load-time) but this can be changed easily in the code if you want to make the refreshes faster / slower. There are also buttons that allow the user to pause the automatic refresh ("Freeze all" / "Unfreeze all"), and hide any of the currency pairings that they do not wish to see displayed. The refresh-pausing functionality allows the user to preserve any zoom or limitation that they have selected via the Plotly viz.

The size of each observation is determined algorithmically using a transformation of the square root of the volume of all orders at that particular price-point calibrated so that the bubbles never become unreasonably large or small. The color-coding allows for easy identification of whales, as described in the section above.

Note: all of these limitations--i.e. the volume minimum, the order book limitations, etc., are parameterized within the app.py code and thus can be easily changed if so desired.

Anyone interested with Python 3.6 installed can download the app.py or clone the repo and run the app locally, just check to be sure you have the few required modules installed (I recommend managing them within a virtual environment created using conda or whatever other package manager you prefer). Once you have Python 3.6 installed, open up a Terminal and type:

pip install -r /path/to/requirements.txt

Once its finished type:

python app.py

Then open up your browser and type in the set of numbers that pop up in terminal, which should look similar to "127.0.0.1:8050".

Support Needed

Below is a summary of the main needs that we have presently with which anyone can assist (this is a community project after all!):

  1. Technical assistance and ideation:

    1. If you have coding experience, check out the code and issues tab and see if you can help with anything or propose new additions.

    2. If you want new features integrated or have any other ideas, open a new issue and I'll address it ASAP.

  2. Overall promotion:

    1. Keep sharing with you friends / fellow traders / coders so that we can get more constructive feedback.

    2. Consider starring us on GitHub as a means of sharing this project with the broader community.

Contribution Rules

All are welcome to contribute issues / pull-requests to the codebase. All I ask is that you include a detailed description of your contribution, that your code is thoroughly-commented, and that you test your contribution locally with the most recent version of the Master branch integrated prior to submitting the PR.

Comments
  • Hosting

    Hosting

    Looking for long term and speedy options for hosting this. I tried Heroku to no avail. The present plan now is to move to AWS long-term, but I'm open to other suggestions in this thread.

    enhancement 
    opened by pmaji 55
  • Better plot coloring / market price visual

    Better plot coloring / market price visual

    • [x] show logic for margins specification and adding background color
    • [ ] need to choose a better background / plot color (just picked one arbitrarily to show proof of concept)
    • [x] need to add horizontal line to show market price
    • [ ] would love to be to selectively color the background such that above the market price is a very faint color of red and below market price is a very faint color of green (this comes after the horizontal line implementation)
    opened by pmaji 35
  • Order Book chart

    Order Book chart

    Hello,

    in my Opinion it would be nice to see the total size of order-book by a line in the background. Due to the fact, we already have the data, it would be little problematic.

    I just have the problem I´m not familiar with Plotty....

    Thanks and Greets

    Theimo

    Edit: Needed Tasks:

    • [x] Sum upp Data for graph
    • [x] Add code for Plotty
    opened by theimo1221 25
  • noob issue, invalid syntax

    noob issue, invalid syntax

    Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32
    
    pip install -r D:\_installers\crypto-whale-watching-app-master\requirements.txt
              ^
    SyntaxError: invalid syntax
    
    opened by shawn-j 15
  • Some orders not grouping together

    Some orders not grouping together

    See: https://i.imgur.com/0UuGiWF.png https://i.imgur.com/pquAdI3.png

    These should all be "ladder whales", 8.000 LTC ladder and 450 BTC ladder, but are 7 and 9 bubbles instead, all for the same amount (1000 LTC and 50 BTC)

    opened by mifunetoshiro 14
  • Some improvements

    Some improvements

    The main gist of https://github.com/pmaji/crypto-whale-watching-app/pull/31 which wasn't merged was to include the total price of a bubble, which I think is valuable information (https://i.imgur.com/8ZwXOIL.png).

    I also changed the TICKERS tuple into a list, because if someone just wants to use for example ETH-USD and removes the rest, Python will consider it as a string and not a tuple, so it will return an error.

    I changed n_unique_orders from float into int because this is always a full number, and saying there are e.g. "5.0" unique orders instead of "5" is redundant.

    I also improved some text formatting and added rounding that @theimo1221 forgot in the ladder whale tooltips (it would say e.g. 3.000000005345 BTC).

    Also changed the server IP from 0.0.0.0 to 127.0.0.1

    opened by mifunetoshiro 9
  • Continuous improvement of documentation

    Continuous improvement of documentation

    • [x] update requirements.txt with few new packages added by contributors
    • [x] figure out what kind of license we want to use for this project and add it to files / potentially read me
    • [x] update README
    • [x] finalize comments throughout
    opened by pmaji 9
  • Adding

    Adding "total price" worth to ladder whale orders

    Now with #66 merged, hovering over a bubble shows the total price of the order/s. I tried to add the same to the ladder whale orders, and if I play with a simple test dataframe it works, but when I try with app.py, I get an error on:

        vol_grp_bid = vol_grp_bid[
            ((vol_grp_bid[TBL_VOLUME] >= minVolume) & (vol_grp_bid['count'] >= 2.0) & (vol_grp_bid['count'] < 70.0))]
    

    for some reason. Could someone with more experience implement this?

    opened by mifunetoshiro 8
  • Questions to explore further beyond the app

    Questions to explore further beyond the app

    Given the treasure trove of data that we now are developing, as well the growing community of traders / coders that are involved in this project, I wanted to open an issue where people can periodically contribute questions that they want explored further.

    My aim to open up a Python Notebook separate from this app in which we can dig into deeper (perhaps more long-form analytic questions) that this app or curiosity in general may create. Comment below with any hypotheses you want tested or theories you want explored, etc., and I will add them to the master list of topics to add to the exploratory Python Notebook that will be created.

    List of subjects to investigate:

    • [ ] Investigate temporal weekly trends (i.e. what really are GDAX's volume / volatility trends over the course of a week, and are these patterns predictable? Do things really go up Sat. night and come down Sunday morning?)
    • [ ] Impact of holidays (if any) on trading patterns
    • [ ] Is there a time-zone that dominates the volume?
    • [ ] Are particularly time-zones more / less bearish / bullish than others?
    • [ ] Is it true that eth price follows btc with a short but perceptible lag?
    • [ ] Correlation between bot activity and price movement (i.e when bot trading is more prominent does that necessarily lead to a higher likelihood of upwards vs. downwards price swings)
    • [ ] Correlation between more volume and price?
    • [ ] Investigation of whether or not TA beats buy-and-hold and over what time span if at all?
    • [ ] Portfolio performance question? Does holding the 20 lowest price coins vs. the 20 highest price coins yield a higher return over time?
    • [ ] Are there any exchanges in particular that more often than not lead the market? (i.e. does an upswing on GDAX cause an upswing elsewhere?)
    • [ ] Look in the data to first identify what seem like the same bot trading regularly, and then see what correlation there is to price swings.
    enhancement 
    opened by pmaji 8
  • Few ideas and questions

    Few ideas and questions

    • Add a link to the bottom of the page linking to the github
    • Add a slider to adjust the refresh time
    • Permit to add dynamically new charts, as we saw, the only thing to change is the eth-btc or eth-usd to get bch-usd etc... It shouldn't be too hard to re-use the code and let user add charts with a button
    • If we could add new charts/pair add a save button and just save the config so we won't loose anything if we stop the app.
    • Adjust the size of the circles size when zoomed in the graph, it's hard to see when you zoom on the small order!

    Q.

    • I was trying to find documentation regarding the column address you request in the code, what does it return ? In the description you said the app allow us to see if walls are from one person of few person right ? In the graph X axis is the price Y the order size, how can I see if the big red circle with a sell order size of 350 and at a price of 815 is only one whale or few others players ? I can only see a circle, what if a guy set a sell order at 815 for 200 and and another one at the same price for 150, how can I see these are two different person/order ?

    Thanks!

    help wanted 
    opened by arsenicks 8
  • Reduction to one callback (All pairs)

    Reduction to one callback (All pairs)

    After long trial and error I found a way to reduce callback and requests to server to 1 regardless of pair amount.

    It also reduces Package Amount to 16 Kb (avg. 4kb per Pair) to before 5Kb per pair.

    Includes #39

    opened by theimo1221 7
  • Does this still work?

    Does this still work?

    When I run app.py (in Pycharm or IntelliJ IDEA Ultimate) I sucessfuly connect to the websockets ( I get " -- Subscribed! --" at least,) but then I get:

    _Exception in thread Thread-4: Traceback (most recent call last): File "C:\Users\user\Miniconda3\envs\envname\lib\threading.py", line 973, in _bootstrap_inner self.run() File "C:\Users\user\Miniconda3\envs\envname\lib\threading.py", line 910, in run self._target(*self._args, **self._kwargs) File "C:\Users\user\Miniconda3\envs\envname\lib\site-packages\cbpro\websocket_client.py", line 41, in _go self._listen() File "C:\Users\userMiniconda3\envs\envname\lib\site-packages\cbpro\websocket_client.py", line 91, in _listen self.on_message(msg) File "C:\Users\user\PycharmProjects\envname\gdax_book.py", line 47, in on_message self.close() File "C:\Users\user\Miniconda3\envs\envname\lib\site-packages\cbpro\websocket_client.py", line 104, in close self.thread.join() File "C:\Users\user\Miniconda3\envs\envname\lib\threading.py", line 1050, in join raise RuntimeError("cannot join current thread") RuntimeError: cannot join current thread Error: messages missing (22854612043 - 22854612022). Re-initializing websocket. Error: messages missing (9076049670 - 9076049429). Re-initializing websocket. _ And it repeats

    Perhaps this might be because I installed the newest versions of zope.interface, MarkupSafe, numpy and pandas, as versions written in the requirements.txt do not seem to be available thorugh pip install?

    Also, during the installation of the requirements there is an error: gdax 1.0.6 & cbpro 1.1.4 require Requests 2.13.0, but 2.21.0 are written in the requirements, which are incompatible

    Thanks for your help

    opened by CryPtoSmartGuy420 0
  • Crypto vs fiat

    Crypto vs fiat

    It would be really interesting see a single chart that shows all crypto-fiat pairs on a single chart. The axes would need to be converted into a single domain like implied BTC market price and order value in USD to make them all accessible on a single plot, but the larger data set might give a clearer picture of the overall status of the market.

    enhancement help wanted good first issue 
    opened by armstrys 1
  • Help wanted with web hosting of app

    Help wanted with web hosting of app

    I installed at

    http://whaledectector.com

    Site missing CSS, shows no graph.

    On console, i get error "No graphs loaded yet".

    screenshot from 2018-09-14 08-09-21

    Any idea why it is not wokring ?

    @pmaji

    enhancement help wanted question 
    opened by serverok 9
  • Adding Binance / Other Exchanges

    Adding Binance / Other Exchanges

    An issue to collect data and tasks needed for adding binance

    • [x] Decrease Callback/ Load Amount to 1 in #40

    • [x] Analyze compatibility Needed changes in Data call? Needed changes in Data storage (Multiple times same pair e.g. "ETH-USD")

    • [x] Discuss form of presentation Original:Send all data to client, and hide/ show selection on clientside with js? Update: Sending all data should be no problem due to changes in #40 Maybe rework show/ hide in the future, to show/ hide complete exchanges.

    • [ ] Add Dropdown to select exchange Update Show/ Hide Button is much easier

    • [ ] define format for get_data function Goal is to keep get_data modular for future.

    • [ ] exclude api calls from get_data function

    When we change this we should change #28 aswell

    Thanks and Greets

    Theimo

    enhancement help wanted good first issue 
    opened by theimo1221 10
Owner
Paul Jeffries
Trained in intl. econ; started in mortgage finance; dabbled in equities & crypto; now working in banking. I enjoy challenging questions regarding value & risk.
Paul Jeffries
A web app to scan crypto markets based on candlestick pattern recognition from

Crypto_Scanner A web app to scan crypto markets based on candlestick pattern recognition from "Japanese Candlestick Charting Techniques: A Contemporar

Chris Qi 27 Jan 1, 2023
A crypto bot that checks the price movement in the markets and creates buy and sell signals

Booter bot Purpose The purpose of this bot is to check the price fluctuations in a given market in binance and create the idealistic signals based on

null 2 Oct 9, 2022
This project is a proof of concept to create a dashboard using Dash to display information about various cryptocurrencies.

This project is a WIP as a way to display useful information about cryptocurrencies. It's currently being actively developed as a proof of concept, and a way to visualize more useful data about various cryptocurrencies.

null 7 Apr 21, 2022
This python module can analyse cryptocurrency news for any number of coins given and return a sentiment. Can be easily integrated with a Trading bot to keep an eye on the news.

Python script that analyses news headline or body sentiment and returns the overall media sentiment of any given coin. It can take multiple coins an

null 185 Dec 22, 2022
Run with one command grafana, prometheus, and a python script to collect and display cryptocurrency prices and track your wallet balance.

CryptoWatch Track your favorite crypto coin price and your wallet balance. Install Create .env: ADMIN_USER=admin ADMIN_PASSWORD=admin Configure you

Rafael Zimmermann 13 Dec 13, 2022
Python Cryptocurrency with stealth addresses

Python Cryptocurrency with stealth addresses. Goal is to have create a cryptocurency that hides transactions totally. I.E. Cant see ammount sent, to who, or from who.

null 3 Aug 4, 2022
Calculate your taxes from cryptocurrency gains

CoinTaxman helps you to bring your income from crypto trading, lending, ... into your tax declaration.

Jeppy 118 Dec 26, 2022
A bot for FaucetCrypto a cryptocurrency faucet. The bot can currently claim PTC ads, main reward and all the shortlinks except exe.io and fc.lc.

A bot for the high paying popular cryptocurrency faucet Faucet Crypto. The bot is built using Python and Selenium, currently it is under active develo

Sourav R S 81 Dec 19, 2022
GreenDoge is a modern community-centric green cryptocurrency based on a proof-of-space-and-time consensus algorithm.

GreenDoge Blockchain Download GreenDoge blockchain GreenDoge is a modern community-centric green cryptocurrency based on a proof-of-space-and-time con

null 40 Sep 11, 2022
Stor is a community-driven green cryptocurrency based on a proof of space and time consensus algorithm.

Stor Blockchain Stor is a community-driven green cryptocurrency based on a proof of space and time consensus algorithm. For more information, see our

Stor Network 15 May 18, 2022
Signarly is a cryptocurrency trading bot.

Signarly is a cryptocurrency trading bot.

Zakaria EL Mesaoudi 5 Oct 6, 2022
SHIBgreen is a cryptocurrency forked from Chia and uses the Proof of Space and Time consensus algorithm

SHIBgreen is a cryptocurrency forked from Chia and uses the Proof of Space and Time consensus algorithm

null 13 Jul 13, 2022
Cryptocurrency with implementet Blockchain

Cryptocurrency with implementet Blockchain

Mario 1 Mar 24, 2022
A Docker image for plotting and farming the Chia™ cryptocurrency on one computer or across many.

An easy-to-use WebUI for crypto plotting and farming. Offers Plotman, MadMax, Chiadog, Bladebit, Farmr, and Forktools in a Docker container. Supports Chia, Cactus, Chives, Flax, Flora, HDDCoin, Maize, N-Chain, Staicoin, and Stor among others.

Guy Davis 328 Jan 1, 2023
This is a simple application to generate HD wallet addresses for cryptocurrency coins.

HD-Wallet-Address This is a mini service to generate addresses in the master HD-Wallet. It will use py_crypto_hd_wallet package as a base. Prerequisit

Amin Abbasi 1 Dec 16, 2021
Cryptocurrency trading bot with a graphical user interface with support for simulations, backtests, optimizations, and running live bots.

Cryptocurrency trading bot with a graphical user interface with support for simulations, backtests, optimizations, and running live bots.

Mihir Shrestha 834 Dec 30, 2022
Aggregate real-time market data from cryptocurrency exchanges, filter, sort and format as TradingView watchlists.

tvbuddy Aggregate real-time market data from cryptocurrency exchanges, filter, sort and format as TradingView watchlists. Developed and tested on Pyth

Ossian Winter 2 Jan 7, 2022
PytoPrice is an automation program to fetch the latest price of a cryptocurrency of your choice at a user-customizable update interval.

PyToPrice (Python Crypto Price) PytoPrice is an automation program to fetch the latest price of a cryptocurrency of your choice at a user-customizable

Peter 1 Jun 16, 2022
💰 An Alfred Workflow that provides current price of cryptocurrency

Coin Ticker for Alfred Workflow An Alfred Workflow that provides current price and status about cryptocurrency from cryptocompare.com. Supports Alfred

Bumsoo Kim (Ian) 14 Nov 17, 2022