binance_harvester
A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins storing data in an SQLite3 database for use by algorithmic and bot traders
The script will populate an SQLite3 database called prices.sqlite3 in the same directory as the script.
By default the script will populate the DB with data from 5 minute candles and takes about 14 seconds from the reception of candle data on the socket to publish the data for every crypto pair and indicator in the tables ready for your algos or bots
The top trending coins list is stored in a table called "top_cryptos"
Other data is stored in tables named for the coin pair eg "BTCUSDT"
coin pairs used are listed in the text file USDT_pairs.txt
The fields of recorded data and indicators are as follows: Timestamp,Open,High,Low,Close,Volume,macd,macd1226,macd_diff,macd_diff1020,macd_diff2550,macd_diff50100,stoch,stochrsi,rsi,sma,ema,chaikin_money_flow,mfi,trix,tema
Dependencies
import websocket, json, pprint
from sqlite3 import connect
import pandas as pd
import sqlite3
import argparse
import time
import math
import os
from pprint import pprint
from datetime import datetime, timedelta, timezone
import csv
from collections import deque
import copy
from binance.client import Client
from binance.enums import *
from ta import add_all_ta_features
from ta.utils import dropna
from ta.trend import macd, macd_diff, sma_indicator, ema_indicator, trix
from ta.momentum import stoch, stochrsi, stoch_signal, rsi
from ta.volume import chaikin_money_flow, money_flow_index
from functools import partial
import multiprocessing
import threading
import os
import time
from operator import itemgetter
import talib
You must also create a file called config.py in the same directory as the script and populate it with your Binance API keys following the format of SAMPLE_config.py
Usage
python binance_harvester -T 5
Ideally run in the background:
nohup python binance_harvester.py -T 5 &
Where the T parameter specifies number of hours for calculating the list of top trending coins eg using "1" will give you the top trending coins over the last 1 hour.
The top trending coins list is stored in a table called "top_cryptos"
Other data is stored in tables named for the coin pair eg "BTCUSDT"
coin pairs used are listed in the text file USDT_pairs.txt
6 Dec 6, 2022
5 Nov 22, 2022
13 Dec 27, 2022
0 Dec 15, 2021
10 Aug 18, 2022
1 Jan 2, 2022
4 Aug 30, 2022
31 Apr 17, 2022
0 Nov 22, 2021
1 Jan 24, 2022
0 Aug 18, 2021
32 Dec 31, 2022
14 Dec 9, 2022
3 Sep 9, 2022
71 Oct 4, 2022
19 Nov 17, 2022
1 Mar 28, 2022
4 Jan 9, 2022