An improved version of the original AutoDD

Overview

About AutoDD Rev 2

AutoDD = Automatically does the "due diligence" for you. If you want to know what stocks people are talking about on reddit, this little program might help you.

Original author - Fufu Fang https://github.com/fangfufu

Rev 2 Author - Steven Zhu https://github.com/kaito1410 Napo2k https://github.com/Napo2k

The original AutoDD produced a simple table of the stock ticker and the number of threads talking about the ticker.

Version 2 of AutoDD adds some options and features that the original did not have.

- ability to display a change in results (ie, an increase or decrease of score from the previous day to today)

- ability to pull additional stock information from yahoo finance (such as open and close price, volume, average volume, etc)

- ability to pull results from multiple subreddits (pennystocks, RobinHoodPennyStocks, stocks, Daytrading, etc)

- added score system to calculate a score for each ticker based on the number of occurrences, DD, Catalyst, or technical analysis flair, and number of upvotes

- Can be run with a windows scheduler to run the program at a set time everyday

Requirements

Python (tested on python 3.8.1) https://www.python.org/downloads/release/python-381/

Pip - python get-pip.py https://phoenixnap.com/kb/install-pip-windows#:~:text=PIP%20is%20automatically%20installed%20with,9%2B%20and%20Python%203.4%2B.

psaw - pip install psaw https://pypi.org/project/psaw/

yahooquery - pip install yahooquery https://pypi.org/project/yahooquery/

tabulate - pip install tabulate https://pypi.org/project/tabulate/

The requirements can be installed by running install_requirements.bat / install_requirements.sh

Running

Watch the setup video here https://www.youtube.com/watch?v=YwfwJYjnBFU

To set up the dependencies on Windows 10

1. Install python 3.8 and make sure you add python to the path variable during installation
2. Run install_requirements.bat, it should open a terminal and install the dependencies
3. If all dependencies are installed successfully, run run_auto_dd.bat
4. After 1-2 minutes, you should find a table_records.txt file in the AutoDD folder
5. To generate a new table, simply run run_auto_dd.bat again, it will append a new table to the table_records.txt file

To set up the dependencies on Linux/MacOSX

1. Install python 3.8 and Pip3 https://medium.com/swlh/installing-python-and-pip-on-mac-72b7639a58
2. Run install_requirements.sh, it should open a terminal and install the dependencies
3. If all dependencies are installed successfully, run run_auto_dd.sh
4. After 1-2 minutes, you should find a table_records.txt file in the AutoDD folder
5. To generate a new table, simply run run_auto_dd.bat again, it will append a new table to the table_records.txt file

For Advanced Users:

1. Simply open the terminal (powershell or command prompt on windows, terminal on linux/MacOSX) and navigate to the AutoDD folder, then type:
	
	python main.py -h
	
2. Follow the help document and set up the optional parameters as you'd like. 

Columns Explained

Code - Ticker Name

Total - Total score on the ticker for r/pennystock subreddit. Higher means more discussions/chatter about this ticker

Recent - Score of the ticker from the last X hours. By default, Recent shows the score from the last 12 hours. If you change the interval for example --interval 48, then recent show data from 24 hours ago (48 divide by 2)

Prev - Score of the ticker from the last X to 2X hour period. By default, Prev shows the score from the last 12-24 hour period. If you change the interval for example --interval 48, then recent show data from the 24-48 hour period

Change - (Recent score - Prev score) Shows increase or decrease in amount of chatter/discussions about this ticker. Positive numbers = increase in discussions, higher numbers means more discussions/chatter about this ticker

Rockets - Number of Rocket Emojis

Price - Current stock price

%DayChange - Percentage change in todays price compared to yesterday

%50DayChange - Percentage change in todays price compared to the last 50 day average

%ChangeVol - Percentage Change in volumn from today to the 3 month average

Float - Float shares, number of tradable shares of the ticker

Industry - Industry of the company if available

Example output

Default Output:

Alt text

Allsub Option Output:

Alt text

Yahoo Option Output:

Alt text

Options

In terminal, type:

python main.py -h

This will produce the following help text:

usage: main.py [-h] [--interval [INTERVAL]] [--min [MIN]] [--adv] [--sub [SUB]] [--sort [SORT]] [--filename [FILENAME]]

AutoDD Optional Parameters

optional arguments:
-h, --help            show this help message and exit
--interval [INTERVAL]
						Choose a time interval in hours to filter the results, default is 24 hours
--min [MIN]           Filter out results that have less than the min score, default is 10
--yahoo               Using this parameter shows yahoo finance information on the ticker, makes the script run slower!
--sub [SUB]           Choose a different subreddit to search for tickers in, default is pennystocks
--sort [SORT]         Sort the results table by descending order of score, 1 = sort by total score, 2 = sort by recent score, 3 = sort by previous score, 4 = sort by change in score, 5 = sort by # of rocket emojis
--allsub              Using this parameter searchs from one subreddit only, default subreddit is r/pennystocks.
--csv                 Using this parameter produces a table_records.csv file, rather than a .txt file
--filename [FILENAME]
						Change the file name from table_records.txt to whatever you wish

Interval (Time interval)

1. Choose a time interval N in hours to filter the results, default is 24 hours

2. The score in the Total column shows the score for each ticker in the last N hours

3. The score in the Recent column shows the score for each ticker in the last N/2 hours, default to 12h

4. The score in the Prev column shows the score for each ticker in the last N/2 - N hours, default is 12h - 24h

5. The score in the other subreddit columns shows the score for each ticker in the last 24 hours

Min (Minimum score)

1. Filter out results that have less than the min score in the Title column, default is 10

Yahoo (Yahoo Finance toggle)

1. Using this parameter shows yahoo finance information, running yahoo mode is slower

2. This options shows additional yahoo information on the ticker, such as open price, day low, day high, forward PE, beta, volume, etc.

Sub (Subreddit Selection)

1. Choose a different subreddit to search for tickers in, default is pennystocks

2. When a different subreddit is choosen, the total, recent, prev columns contain the score for the choosen subreddit

3. Possible choices: pennystocks, RobinHoodPennyStocks, Daytrading, StockMarket, stocks

Sort

1. Sort the results by descending order of score, by default the table shows the highest total score first

2.  pass in values 1, 2, 3, or 4

3. 1 = sort by total score, 2 = sort by recent score, 3 = sort by previous score, 4 = sort by change in score, 5 = sort by change in # of rocket emojis

Allsub (Subreddit toggle)

1. Using this parameter shows scores on the other subreddits such as RobinHoodPennyStocks, Stocks, WallStreetBets, etc

Csv

1. Outputs table_records.csv file

Filename

1. choose a different filename, this programs saves the table results to table_records.txt in the same folder as the AutoDD.py program

Troubleshoot

ModuleNotFoundError: No module named 'something'

- This means the dependency was not installed correctly, try running: 

	pip install 'something'
	
- Another possibility is that python is using the wrong version, try:
	
	python3 main.py

AutoDD.py not found

- This means the terminal can't find the python script, either navigate to the AutoDD folder using terminal or

	python path-to-autodd-folder/AutoDD.py
	ie. python C:/AutoDD_Folder/AutoDD.py

Scheduler (Tested on Windows)

  1. Create a .bat file and type in:

    python path-to-AutoDD\AutoDD.py --whatever options you want to configure

  2. Open windows Task Scheduler

  3. Create a basic task

  4. Fill in the name and description

  5. Choose a trigger that works for you, mine is every day

  6. Choose "Start a program" and put in the path to your .bat file

    • ie. "C:\AutoDD-folder\run_auto_dd.bat"
  7. That's it, just check table_records.txt or the file name that you've selected and it will have the table ready

Developers/Advanced Users

I'm a C++ main, so excuse my python code/inefficencies with handling tables and lists in python.

I've put a couple global variables for some advanced users to allow for easy modifications:

# dictionary of possible subreddits to search in with their respective table column name
subreddit_dict = {'pennystocks' : 'pnystks',
				  'RobinHoodPennyStocks' : 'RHPnnyStck',
				  'Daytrading' : 'daytrade',
				  'StockMarket' : 'stkmrkt',
				  'stocks' : 'stocks'}

# dictionary of ticker financial information to get from yahoo
financial_measures = {'currentPrice' : 'Price', 'quickRatio': 'QckRatio', 'currentRatio': 'CrntRatio', 'targetMeanPrice': 'trgtmean', 'recommendationKey': 'recommadtn'}

# dictionary of ticker summary information to get from yahoo
summary_measures = {'previousClose' : 'prvCls', 'open': 'open', 'dayLow': 'daylow', 'dayHigh': 'dayhigh', 'payoutRatio': 'pytRatio', 'forwardPE': 'forwardPE', 'beta': 'beta', 'bidSize': 'bidSize', 'askSize': 'askSize', 'volume': 'volume', 'averageVolume': 'avgvolume', 'averageVolume10days': 'avgvlmn10', 'fiftyDayAverage': '50dayavg', 'twoHundredDayAverage': '200dayavg'}


# note: the following scoring system is tuned to calculate a "popularity" score
# feel free to make adjustments to suit your needs

# x base point of for a ticker that appears on a subreddit title or text body that fits the search criteria
base_points = 4

# x bonus points for each flair matching 'DD' or 'Catalyst' of for a ticker that appears on the subreddit
bonus_points = 2

# every x upvotes on the thread counts for 1 point (rounded down)
upvote_factor = 2	

License

AutoDD - Automatically does the "due diligence" for you. 
Copyright (C) 2020  Fufu Fang, kaito1410, Napo2k

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see .
Comments
  • Running with -sub wallstreetbets slow

    Running with -sub wallstreetbets slow

    Hi there,

    First of all thank you for your excellent work.

    Running main.py with wallstreetbets sub option is incredibly slow. My thought is that because of the huge dataset, but processing one entry a second is not ideal.

    Thank you frou your app.

    opened by marcellelek 7
  • undercounting of symbols?

    undercounting of symbols?

    I just went through your code, I love it and will try my hand at using it tomorrow.

    I just found what seems to be an undercounting of tickers:

            # title_extracted is a set, duplicate tickers from the same title counted once only
            for k in title_extracted:
    
                if k in all_dict:
                    all_dict[k] += increment
                else:
                    all_dict[k] = increment
    
            # avoid counting additional point for the tickers found in the text body
            # only search the text body if ticker was not found in the title
            if len(title_extracted) > 0:
                    continue
    
            for m in selftext_extracted:
    
                if m in all_dict:
                    all_dict[m] += increment
                else:
                    all_dict[m] = increment
    

    Doesn't the above miss out on instances when a ticker appears in the selftext but not in the title? (eg say the title contains AAPL and the selftext contains MSFT, it would skip MSFT).

    Since rocket_tickers is already a set, shouldn't the above be replaced by the following?

            for k in rocket_tickers:
    
                if k in all_dict:
                    all_dict[k] += increment
                else:
                    all_dict[k] = increment
    

    Thanks again for sharing this code.

    opened by gobbedy 5
  •     from psaw import PushshiftAPI ImportError: No module named psaw

    from psaw import PushshiftAPI ImportError: No module named psaw

    Hi, I have a mac, everything is working until I run_auto_dd. sh and get this error: from psaw import PushshiftAPI ImportError: No module named psaw I did isntall all requirements multiple times and have python 3.

    thank you!

    opened by gmct05 5
  • issue with rocket count?

    issue with rocket count?

    Hello,

    rockets seem to be counted in an odd way. Let's take an example. Suppose a reddit submission has the following title: "PLEASE BUY AAPL, MSFT, GME NOW"

    then for that submission you would have

    rocket_dict['PLEASE'] += 6
    rocket_dict['BUY'] += 6
    rocket_dict['AAPL'] += 6
    rocket_dict['MSFT'] += 6
    rocket_dict['GME'] += 6
    rocket_dict['NOW'] += 6
    

    Shouldn't it be +=1 for each one?

    Could you please clarify what rockets count?

    PS. I should have finished the refactored code by Monday or so if you're interested. Speedup is in the order of 100x when there are many tickers.

    opened by gobbedy 4
  • Returning Empty Tables

    Returning Empty Tables

    I am not sure what happened. The program was working perfectly until I tried to use it today. Now it just returns nothing. I attempted a new install and it still reproduced the same result. There are no error logs so it's a little difficult to isolate the problem.

    Screen Shot 2021-01-27 at 12 24 46 PM

    Has Reddit changed something on their end?

    opened by TheSwindler44 4
  • Issue running on Ubuntu/macOS

    Issue running on Ubuntu/macOS

    Hey there,

    I'm getting the following error trying to run the main.py on both Ubuntu and macOS.

    Traceback (most recent call last):
      File "main.py", line 101, in <module>
        main()
      File "main.py", line 98, in main
        print_tbl(results_tbl, args.filename, args.allsub, args.yahoo, args.csv)
      File "/app/auto-dd/AutoDD.py", line 378, in print_tbl
        writer.writerow(row)
    UnicodeEncodeError: 'ascii' codec can't encode character '\u2014' in position 87: ordinal not in range(128)
    

    Do you have any idea on what I can do to get it running? It seems to have trouble printing the table after gathering the data. I can get it running on Windows though.

    Thanks in advance!

    opened by garygrossgarten 3
  • redundancy?

    redundancy?

    Am I right that or row[1][1] >= min_val is redundant in the code below?

    tbl = [row for row in tbl if row[1][0] >= min_val or row[1][1] >= min_val]
    

    If I'm not mistaken row[1][0] is the total score (prev + current) for the ticker, and row[1][1] is the current score.

    I'm removing the redundant part in my version of the code, just making sure I haven't misunderstood something?

    opened by gobbedy 2
  • Error Code 429 pushshift.io

    Error Code 429 pushshift.io

    On first time running, recieve the following:

    C:\Users*\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py:192: UserWarning: Got non 200 code 429 warnings.warn("Got non 200 code %s" % response.status_code) C:\Users\Shokkatweej\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py:180: UserWarning: Unable to connect to pushshift.io. Retrying after backoff. warnings.warn("Unable to connect to pushshift.io. Retrying after backoff.")

    And then stalls at 'Getting quick stats...'

    opened by shokkakhan 2
  • Division by zero exception found

    Division by zero exception found

    Hi,

    found this exception by running script today(13:00 CET, 2020/1/26 on Win10).

    Getting submissions...
    Searching for tickers...
    Populating results...
    Counting rockets...
    Getting quick stats...
    Traceback (most recent call last):
      File "main.py", line 101, in <module>
        main()
      File "main.py", line 92, in main
        results_tbl = getQuickStats(results_tbl)
      File "C:\Users\Rado\Desktop\AutoDD\AutoDD_Rev2-main\AutoDD.py", line 489, in getQuickStats
        change_50day = ((float(price) - float(avg50day))/float(avg50day))*100
    ZeroDivisionError: float division by zero
    

    My temporal "fix" was:

     if price is not None and price != 0:
                    if avg50day > 0:
                        change_50day = ((float(price) - float(avg50day))/float(avg50day))*100
                    else:
                        change_50day = 0
    

    on line 489 in AutoDD.py

    opened by radoslavpalenik 2
  • Table Columns Incorrect

    Table Columns Incorrect

    python3 main.py --allsub

    Maybe I am missing something in the documentation but I don't believe the numbers in specific columns are adding up correctly. For example, the price of ZOM has never been 186. (It looks like price is under Industry???)

    Is this a bug?

    Screen Shot 2021-01-13 at 3 17 10 PM
    opened by TheSwindler44 2
  • Definition of key_stats_measures is missing

    Definition of key_stats_measures is missing

    It looks like in the latest code the definition of key_stats_measures dict disappeared which is causing an error when running. I added it back in at lines 64/65 and everything worked as it should.

    opened by peterjolles 1
Owner
Steven Zhu
Steven Zhu
The sequel to SquidNet. It has many of the previous features that were in the original script, however a lot of the functions that do not serve much functionality have been removed.

SquidNet2 The sequel to SquidNet. It has many of the previous features that were in the original script, however a lot of the functions that do not se

DrSquidX 5 Mar 25, 2022
TLD records archive. Revisiting the original TLDR project by mandatoryprogrammer, on the hunt for more root nameserver changes.

tldr A(nother) continuously updated historical TLD records archive. This repository is updated approximately every three hours with the results from D

Chris Partridge 11 Dec 14, 2022
BibleNotifyDesktop - Desktop version of Bible Notify

Bible Notify Desktop This is the repository for the Desktop version of the daily

Bible Notify 5 Nov 16, 2022
A updated and improved version from the original Discord-Netflix from Nirewen.

Discord-Netflix A updated version from the original Discord-Netflix from nirewen A Netflix wrapper that uses Discord RPC to show what you're watching

Void 42 Jan 2, 2023
A modern, easy to use, feature-rich, and async ready API wrapper improved and revived from original discord.py.

A Python API wrapper that is improved and revived from the original discord.py

Orion 19 Nov 6, 2021
Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of images as "pixels"

picinpics Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of

RodrigoCMoraes 1 Oct 24, 2021
FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

XCL 191 Dec 31, 2022
One version package to rule them all, One version package to find them, One version package to bring them all, and in the darkness bind them.

AwesomeVersion One version package to rule them all, One version package to find them, One version package to bring them all, and in the darkness bind

Joakim Sørensen 39 Dec 31, 2022
This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

신재욱 17 Sep 25, 2022
[Open Source]. The improved version of AnimeGAN. Landscape photos/videos to anime

[Open Source]. The improved version of AnimeGAN. Landscape photos/videos to anime

CC 4.4k Dec 27, 2022
The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.

Face Alignment in Full Pose Range: A 3D Total Solution By Jianzhu Guo. [Updates] 2020.8.30: The pre-trained model and code of ECCV-20 are made public

Jianzhu Guo 3.4k Jan 2, 2023
An improved version of the common ˙pacman -S˙

BetterPacmanLook An improved version of the common pacman -S. Installation I know that this is probably one of the worst solutions and i will be worki

null 1 Nov 6, 2021
Generative Exploration and Exploitation - This is an improved version of GENE.

GENE This is an improved version of GENE. In the original version, the states are generated from the decoder of VAE. We have to check whether the gere

null 33 Mar 23, 2022
Improved version calculator, now using while True and etc

CalcuPython_2.0 Olá! Calculadora versão melhorada, agora usando while True e etc... melhorei o design e os carai tudo (rode no terminal, pra melhor ex

Scott 2 Jan 27, 2022
A tool used to encrypt Python scripts version < 2.7 and version < 3.9

A tool used to encrypt Python scripts version < 2.7 and version < 3.9

Fajar Kim 1 Dec 14, 2021
MCRPC (Minecraft Resource Pack Comparator) checks your resource pack against any version of Minecraft to show resources missing from your pack for that version.

Minecraft Resource Pack Comparator MCRPC checks your resource pack against any version of Minecraft to show resources missing from your pack for that

null 3 Nov 3, 2022
A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Neural 3D Mesh Renderer in PadddlePaddle A PaddlePaddle version of Neural Renderer, refer to its PyTorch version Install Run: pip install neural-rende

AgentMaker 13 Jul 12, 2022
Modern responsive template for the Django admin interface with improved functionality. We are proud to announce completely new Jet. Please check out Live Demo

Django JET Modern template for Django admin interface with improved functionality Attention! NEW JET We are proud to announce completely new Jet. Plea

Geex Arts 3.4k Dec 29, 2022
Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

RGF-team 364 Dec 28, 2022