Hi!

I am unable to use the utils.py file since the "wr" (line 258) and "money_flow_index" (line 271) functions are not defined anywhere.

When you're using time series analysis and technical indicators, you can't shuffle your data. Real time series data is not shuffled. Also, shuffling the data leaks historical data, as the technical indicators are calculated based on historical data. Your first indicator should have been your very high accuracy (i.e. if you could accurately predict 85%+ accuracy against the stock market, you'd be a rich man).

This is a very similar issue with this research: https://www.reddit.com/r/algotrading/comments/cv83yh/overfitting

running stock_cnn.py ['/content/stock_cnn_blog_pub/src/train.py', 'WMT', 'original'] ../stock_history/WMT ../outputs/fresh_rolling_train ../stock_history/WMT/WMT.csv 1 ) 03-11-2020 13_14_44 MainThread INFO\ Initialized logging at path ../outputs/fresh_rolling_train/logs/log_WMT_original_fresh_rolling_train_03-11-2020_13_14_44.log Tensorflow devices path to company data: ../stock_history/WMT/WMT.csv 2 ) 03-11-2020 13_14_44 MainThread DEBUG\ Data for WMT ready to use 3 ) 03-11-2020 13_14_44 MainThread DEBUG\ Technical indicators already calculated. Loading... 4 ) 03-11-2020 13_14_44 MainThread DEBUG\ Dropped 26 nan rows before label calculation 5 ) 03-11-2020 13_14_44 MainThread DEBUG\ creating label with original paper strategy Calculating labels 0% 0/5260 [00:00<?, ?it/s]

IndexError Traceback (most recent call last) /content/stock_cnn_blog_pub/src/train.py in () 71 72 start_time = time.time() ---> 73 data_gen = DataGenerator(company_code, PATH_TO_COMPANY_DATA, OUTPUT_PATH, strategy_type, False, logger) 74 # exit here, since training is done with stock_keras.ipynb. 75 # comment sys.exit() if you want to try out rolling window training.

2 frames /content/stock_cnn_blog_pub/src/data_generator.py in init(self, company_code, data_path, output_path, strategy_type, update, logger) 40 self.end_col = 'eom_26' 41 self.update = update ---> 42 self.download_stock_data() 43 self.df = self.create_features() 44 self.feat_idx = self.feature_selection()

/content/stock_cnn_blog_pub/src/data_generator.py in create_features(self) 266 267 prev_len = len(df) --> 268 df.dropna(inplace=True) 269 df.reset_index(drop=True, inplace=True) 270 self.logger.append_log("Dropped {0} nan rows after label calculation".format(prev_len - len(df)))

/content/stock_cnn_blog_pub/src/data_generator.py in create_labels(self, df, col_name, window_size) 142 if max_index == window_middle: 143 labels[window_middle] = 0 --> 144 elif min_index == window_middle: 145 labels[window_middle] = 1 146 else:

IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

I have type casted the window_middle as int but still this error throws up no matter what I try to change.

Hey,

Thanks so much for sharing your code by the way and all the effort!

I am setting these options manually in the script. However, I have not figured out what are the options for strategy_type yet. Can you elaborate on this perhaps?

in train.py, line 45: #company_code = args[1] #strategy_type = args[2] company_code = 'HPQ' strategy_type = ''

thanks in advance, Alex

Hello!

Thanks for you article and repo. I'm really interested in it! Could you explain something?

At the and of your article https://towardsdatascience.com/stock-market-action-prediction-with-convnet-8689238feae3 you wrote results: https://miro.medium.com/max/403/1*YLKDhlcx6TtzIoVSdJ2xNw.png For example, for class 0: TP 49 FP 195.

But, in https://www.kaggle.com/darkknight91/predicting-stock-buy-sell-signal-using-cnn/ in the end: [[ 53 0 11] [ 0 50 11] [ 38 53 784]] TP 53 and only 38 FP !!! It's fantastic result!

I tried all params from your example and I can easily get first result with FP = 4*TP, but didn't get even close to TP =~ FP Val_loss is too big and f1_metric poor (( How did you manage to get second results? This results is really promising!

Thank you!

Hi,

I am getting the following errors, when executing

history = model.fit(x_train, y_train, epochs=params['epochs'], verbose=0,
                    batch_size=64, shuffle=True,
                    validation_data=(x_cv, y_cv),
                    callbacks=[es, mcp, rlp]
                    , sample_weight=sample_weights)

returns:

"WARNING:tensorflow:Early stopping conditioned on metric val_loss which is not available. Available metrics are: WARNING:tensorflow:Can save best model only with val_loss available, skipping. WARNING:tensorflow:Reduce LR on plateau conditioned on metric val_loss which is not available. Available metrics are: lr"

Do you know why this might be happening?

best wishes and thanks, Alex

When I run the first cell of the notebook stock_keras.ipynb, i got the following error. This is appearing after doing all the labels for every technical indicators except the IBR.

Here is the context : running stock_cnn.py ['./train.py', 'WMT', 'original'] ../stock_history/WMT ../outputs/fresh_rolling_train ../stock_history/WMT/WMT.csv 1 ) 28-07-2020 16_40_55 MainThread INFO\ Initialized logging at path ../outputs/fresh_rolling_train/logs/log_WMT_original_fresh_rolling_train_28-07-2020_16_40_55.log Tensorflow devices path to company data: ../stock_history/WMT/WMT.csv 2 ) 28-07-2020 16_40_55 MainThread DEBUG\ Data for WMT ready to use 3 ) 28-07-2020 16_40_55 MainThread DEBUG\ Technical indicators already calculated. Loading... 4 ) 28-07-2020 16_40_55 MainThread DEBUG\ Dropped 26 nan rows before label calculation 5 ) 28-07-2020 16_40_55 MainThread DEBUG\ creating label with original paper strategy Calculating labels HBox(children=(IntProgress(value=0, max=5007), HTML(value='')))

IndexError Traceback (most recent call last) ~/Downloads/stock_cnn_blog_pub-master/src/train.py in 71 72 start_time = time.time() ---> 73 data_gen = DataGenerator(company_code, PATH_TO_COMPANY_DATA, OUTPUT_PATH, strategy_type, False, logger) 74 # exit here, since training is done with stock_keras.ipynb. 75 # comment sys.exit() if you want to try out rolling window training.

~/Downloads/stock_cnn_blog_pub-master/src/data_generator.py in init(self, company_code, data_path, output_path, strategy_type, update, logger) 40 self.update = update 41 self.download_stock_data() ---> 42 self.df = self.create_features() 43 self.feat_idx = self.feature_selection() 44 self.one_hot_enc = OneHotEncoder(sparse=False, categories='auto')

~/Downloads/stock_cnn_blog_pub-master/src/data_generator.py in create_features(self) 266 short = self.strategy_type.split('')[0] 267 long = self.strategy_type.split('')[1] --> 268 df['labels'] = self.create_label_short_long_ma_crossover(df, 'close', short, long) 269 else: 270 df['labels'] = self.create_labels(df, 'close')

~/Downloads/stock_cnn_blog_pub-master/src/data_generator.py in create_labels(self, df, col_name, window_size) 142 143 labels[window_middle] = 1 --> 144 else: 145 146 labels[window_middle] = 2

IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices