Hi! I was wondering if there's a method to append new data. I would like to do something like that:

new_frame = pd.DataFrame(data=[[date, open, high, low, close]], columns=['DateTime', 'Open', 'High', 'Low', 'Close'])
Data = Data.append(new_frame, ignore_index=True)

This code will throw you an exception because you have to marge the different columns. But, it would be great if we just required to add the OHLC data so that the StockDataFrame automatically recalculates the statistics and indicators. Are there different alternatives to append new data?

Thanks in advance,

my code:

    @classmethod
    def get_atr(cls, candles):
        stock = StockDataFrame.retype(TCandle.to_df(candles))
        return list(stock.get('atr'))

I got the following warning:

  /Users/yurenji/.conda/envs/tangle/lib/python3.6/site-packages/pandas/core/indexing.py:189: SettingWithCopyWarning: 
  A value is trying to be set on a copy of a slice from a DataFrame
  
  See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
    self._setitem_with_indexer(indexer, value)

-- Docs: http://doc.pytest.org/en/latest/warnings.html

I tried 'macdh' and 'sma', they don't have this issue.

I have added Volume Weighted Average Price indicator support

my knowledge of vwap is based on

code sample:- stocks = StockDataFrame.retype(df) print(stocks['vwap'].tail(10))

gives output:- Datetime 2020-12-09 12:15:00+05:30 935.595807 2020-12-09 12:20:00+05:30 935.566596 2020-12-09 12:25:00+05:30 935.548274 2020-12-09 12:30:00+05:30 935.543816 2020-12-09 12:35:00+05:30 935.539725 2020-12-09 12:40:00+05:30 935.529548 2020-12-09 12:45:00+05:30 935.516953 2020-12-09 12:50:00+05:30 935.502845 2020-12-09 12:55:00+05:30 935.490450 2020-12-09 12:59:09+05:30 935.490450 Name: vwap, dtype: float64

The indicator was tested against the reference investopedia article. It has more settings than other indicators, so the name could be parsed into five parts now.

close_10_kama_2_30

has three settings:

• 10 is the number of periods for the Efficiency Ratio (ER).
• 2 is the number of periods for the fastest EMA constant.
• 30 is the number of periods for the slowest EMA constant.

To make sure regular indicators are parsed as usual, only those within the tuple MULTI_SPLIT_INDICATORS will be parsed into five parts.

are TEMA and TRIX adjustable like other metrics such as .get('tema_20') || .get('trix_35')

Couldn't find on stackoverflow or any other forums so I am at the source : D

Awesome lib!

Hey,

Wondering if somebody could help a guy out in getting this installed and setup up to use? I'm pretty new at this so any advice would be wonderful.

I have python installed, downloaded the files and I researched that I can get the "pandas.DataFrame" through Anaconda. Not sure where to go from here.

Thanks

I think this would be an interesting feature that I'd be willing to help out with.

Ideally you could call a function similar to df.get_stock_stats() that returns a data frame with every single indicator.

any advice?

I'm trying to get intraday macdh data, so did the following:

Example of df before doing anything:

def get_macdh(x):
    col = x.get('macdh')
    # do something with macdh
    result = True if col is not None else False
    return result

stock.groupby(['date','ticker']).apply(get_macdh)

Result:

date        ticker
2021-02-08  AAPL      False
            FB        False
2021-02-09  AAPL      False
            FB        False
2021-02-10  AAPL      False
            FB        False
2021-02-11  AAPL      False
            FB        False
2021-02-12  AAPL      False
            FB        False
2021-02-16  AAPL      False
            FB        False
2021-02-17  AAPL      False
            FB        False

It appears no matter what i try to do, macdh isnt getting initialized within the groupby? If i do a stock.get('macdh') before, the values will be wrong as itll include previous day figures as well.

Hi,

Pandas recently released 1.0.0 and since then, rsi calculations seem to run into KeyError for some reason.

Here's very simple sample code:

#!/usr/bin/env python3

import pandas_datareader.data as web
from stockstats import StockDataFrame

spx = web.DataReader('^GSPC', 'yahoo')
dataframe = StockDataFrame.retype(spx)
print(dataframe['rsi_14']['2020-01-31'])

With pandas 0.25.3, it works fine:

Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 int-date-0.1.8 lxml-4.5.0 numpy-1.18.1 pandas-0.25.3 pandas-datareader-0.8.1 python-dateutil-2.8.1 pytz-2019.3 requests-2.22.0 six-1.14.0 stockstats-0.3.0 urllib3-1.25.8
$ ./test.py
venv/lib/python3.6/site-packages/pandas/core/indexing.py:205: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_with_indexer(indexer, value)
43.43522572721968

However, with pandas 1.0.0, KeyError occurs:

Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 int-date-0.1.8 lxml-4.5.0 numpy-1.18.1 pandas-1.0.0 pandas-datareader-0.8.1 python-dateutil-2.8.1 pytz-2019.3 requests-2.22.0 six-1.14.0 stockstats-0.3.0 urllib3-1.25.8
$ ./test.py 
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_with_indexer(indexer, value)
Traceback (most recent call last):
  File "venv/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1622, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'rsi_14'

The problem could be on pandas side, though they bumped the major version so I suspect stockstats might want to support retyping of pandas 1.0.0 and on as well.

Hi,

there are cases where the kdj columns are all NaN. This seems to be related to rsv_9 beeing NaN.

Here is sample data (the first 20 lines of ASML stock quote) which shows the problem:

date        adj_close     close      high       low      open     volume
2010-01-01   29.28827  23.99998  23.99998  23.99998  23.99998        0.0
2010-01-04   29.60560  24.26000  24.31999  23.89002  23.90003  1563900.0
2010-01-05   29.66047  24.30497  24.62498  24.05996  24.11501  1550300.0
2010-01-06   29.97780  24.56500  24.56500  24.18000  24.20503  1133900.0
2010-01-07   29.42866  24.11501  24.45004  23.80001  24.45004  2648700.0
2010-01-08   28.44013  23.30497  24.08498  23.27502  23.99998  3064200.0
2010-01-11   27.37239  22.43002  23.45497  22.43002  23.31999  4640500.0
2010-01-12   27.82389  22.80001  22.92998  22.60004  22.65001  3098000.0
2010-01-13   28.28762  23.18000  23.39999  22.75003  22.80001  3732600.0
2010-01-14   28.26319  23.15998  23.59996  23.09499  23.46998  1851800.0
2010-01-15   27.89709  22.85999  23.43002  22.69498  23.28996  2738400.0
2010-01-18   27.93985  22.89503  22.98504  22.62499  22.90003  1132900.0
2010-01-19   27.67129  22.67496  22.91997  22.62499  22.80501  2392200.0
2010-01-20   28.53165  23.37997  23.87000  22.87501  22.98997  6490400.0
2010-01-21   29.00759  23.76998  24.06997  23.65001  23.74503  4068900.0
2010-01-22   28.53165  23.37997  23.82003  23.33500  23.53998  3842600.0
2010-01-25   27.78725  22.76998  23.18501  22.69998  22.70999  3091000.0
2010-01-26   28.34861  23.22998  23.37004  22.81002  22.99998  2716300.0
2010-01-27   28.12290  23.04502  23.13503  22.71500  23.08999  2130900.0
2010-01-28   27.86656  22.83497  23.73001  22.83497  23.44003  3445800.0

if rsv_9 is calculated with stockstats, the first value is NaN. This leads to all kdj columns beeing NaN as well. I guess this is due to a division by zero error in _get_rsv in line 251:

df[column_name] = ((df['close'] - low_min) /
                   (high_max - low_min).astype('float64') * 100)  

How should the code be modified so that this bug doesn't appear? (or what's the correct value for rsv if high_max - low_min == 0?)

I want to use other values for example in bollinger I would use 8 periods... how is the correct way to do it?

just do a "Sdf.BOLL_PERIOD = 8" ?

I'm glad to see this fantastic work resume updating. Excellent work, and many thanks.

It seems that all the comparison operators have been removed since commit 68f105de6019525b8e940ae369eed308d1065ec5, although I think they are quite an important feature in my case (ee.g. kdjj_0_le_15_c).

Is there any way to do the same thing in the new version? Or do I have to do this comparison by myself?

Anyway, there are still "compare: le, ge, lt, gt, eq, ne" in the new readme file, so I think there might be something going on...

When user specify a customized window for the energy index, it should appear in the column name. The customized column should not overwrite the column with the default window size.

Adjusting supertrend evaluation by comparing previous close value instead of current

Link to the issue (I am unable to link them direcly over github).

Hello, I have switched my supertrend calculations from pandas_ta and I observe a inconsistency in supertrend values. The issue is (probably) caused by specific candle wicks.

I will continously work the description of this issue, since it's a bit difficult to be to debug or even describe.

Current behaviour

The issue occurs at 08:45 or 08:46. The extreme weird looking candles happens and the supertrend flips for a period of one candle.

• Dataset: Binance OHCL BTCUSDT
• Time range: from 2020-09-14 to 2020-09-15 (UTC)
• ST Multiplier: 2
• ST Window length: 25
• Python 3.10
• Stockstats 0.4.1

Note: I am using an web UI which is displaying only "active" supertrend. This is why is the other line missing from the chart. However I am going to provide charts with both lines, but it's a bit confusing a bit since it's not clear which one is "active".

Code snippet

import pandas as pd
import stockstats

filename = 'binance_btcusdt_ohcl_1m.parquet'
ohcl = pd.read_parquet(filename, engine="fastparquet")
st_a = stockstats.wrap(ohcl.copy())

st_a.SUPERTREND_MUL = 4
st_a.SUPERTREND_WINDOW = 25
st_a = st_a[['supertrend', 'supertrend_ub', 'supertrend_lb']]
# Here I merge the st_a with datetime column generated before the ST calculation
# st_a.insert(1, "datetime", date)
# And renaming columns in order to increase readability
# st_b.rename(columns={'supertrend_ub': 'st_upper', 'supertrend_lb': 'st_lower'}, inplace=True)

Supertrend values (raw)

id datetime st value st upper st lower
611587 2020-09-14 08:40:00  10447.284218  10447.284218  10388.507576
611588 2020-09-14 08:41:00  10447.284218  10447.284218  10388.507576
611589 2020-09-14 08:42:00  10447.284218  10447.284218  10388.507576
611590 2020-09-14 08:43:00  10447.025543  10447.025543  10388.507576
611591 2020-09-14 08:44:00  10447.025543  10447.025543  10388.507576
611592 2020-09-14 08:45:00  10388.507576  10393.494348  10388.507576
611593 2020-09-14 08:46:00  10399.716374  10399.716374  10388.507576
611594 2020-09-14 08:47:00  10399.716374  10399.716374  10293.768281
611595 2020-09-14 08:48:00  10399.716374  10399.716374  10293.768281
611596 2020-09-14 08:49:00  10399.716374  10399.716374  10293.768281
611597 2020-09-14 08:50:00  10399.716374  10399.716374  10293.768281

OHCL with supertrend (stockstats)

OHCL with supertrend (pandas_ta)

OHCL with supertrend and both [upper and lowes] values displayed (stockstats)

Used data source - binance_btcusdt_ohcl_1m.parquet I am using Pandas to work with data, loading parquet is very easy - guide - however in case of any issues, I can export data in different format (JSON etc.) binance_btcusdt_ohcl_1m.parquet.zip

Expected behaviour

I do not think that the supertrend - trend should end (change). I think it should continue since the bull (positive/long) candle did not broke it.

Hi there,

thanks for your wonderful lib, really like it.

When using your lib together with other frameworks I run into the problem that a copy call in the other framework caused the column names to be renamed to lower case, which the other framework can't handle. I scanned your code and found that the copy function has this "feature" in it.

As I think a copy should not modify the data compared to the original one I introduced a lowerCase-Flag in wrap and retype causing copy to leave the column names as they are. With that modification I was able to run your StockDataFrame also with other frameworks like backtesting.py

Best regards, Neutro2

Thanks for sharing this work. I'm trying to adapt to it and I just found two problems. My DataFrame is

df = yf.download("AAPL", start="2020-01-01", end="2020-12-31") stock_df = StockDataFrame.retype(df)

when executing

tp = stock_df['middle'] stock_df['res'] = stock_df['middle'] > df['close'] stock_df[['middle', 'close', 'res', 'res_-10_c']]

it returns the error

`-------------------------------------------------------------------------- KeyError Traceback (most recent call last) ~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in getitem(self, item) 1249 try: -> 1250 result = wrap(super(StockDataFrame, self).getitem(item)) 1251 except KeyError:

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/pandas/core/frame.py in getitem(self, key) 3463 key = list(key) -> 3464 indexer = self.loc._get_listlike_indexer(key, axis=1)[1] 3465

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/pandas/core/indexing.py in _get_listlike_indexer(self, key, axis) 1313 -> 1314 self._validate_read_indexer(keyarr, indexer, axis) 1315

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/pandas/core/indexing.py in _validate_read_indexer(self, key, indexer, axis) 1376 not_found = list(ensure_index(key)[missing_mask.nonzero()[0]].unique()) -> 1377 raise KeyError(f"{not_found} not in index") 1378

KeyError: "['res_-10_c'] not in index"

During handling of the above exception, another exception occurred:

IndexError Traceback (most recent call last) /tmp/ipykernel_7091/1854355926.py in 2 tp = stock_df['middle'] 3 stock_df['res'] = stock_df['middle'] > df['close'] ----> 4 stock_df[['middle', 'close', 'res', 'res_-10_c']]

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in getitem(self, item) 1253 if isinstance(item, list): 1254 for column in item: -> 1255 self.__init_column(column) 1256 else: 1257 self.__init_column(item)

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in __init_column(self, key) 1244 self[key] = [] 1245 else: -> 1246 self.__init_not_exist_column(key) 1247 1248 def getitem(self, item):

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in __init_not_exist_column(self, key) 1229 c, r, t = ret 1230 func_name = 'get{}'.format(t) -> 1231 getattr(self, func_name)(c, r) 1232 elif len(ret) == 2: 1233 c, r = ret

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in get_c(self, column, shifts) 251 """ 252 column_name = '{}{}_c'.format(column, shifts) --> 253 shifts = self.get_int_positive(shifts) 254 self[column_name] = self[column].rolling( 255 center=False,

~/anaconda3/envs/yfinance1/lib/python3.9/site-packages/stockstats.py in get_int_positive(self, windows) 958 window = self.to_int(windows) 959 if window <= 0: --> 960 raise IndexError("window must be greater than 0") 961 return window 962

IndexError: window must be greater than 0`

I will appreciate help to solve this problem.