Creepypasta - Text Generator
Website which uses Deep Learning to generate horror stories.
View Demo · View Website Repo · Report Bug · Request Feature
About The Project
Creepypasta Website Demo
There are two parts to the project. One is the Deep Learning model which generates the text. The other is the website which uses the model to generate text.
I have used Deep Learning model to generate text. It is a Neural Network which uses Recurrent Neural Network to generate text.
I have hosted a model on Algorithmia and used it's API to generate text.
Built With
What I learned
- How to clear raw data to use it for training.
- How to use Deep Learning model to generate text.
- How to use Algorithmia to generate text.
- How to use Vercel to deploy a website.
- How to use TensorFlow to train a model.
- How to train a model incrementally.
- How to handle large data.
- How to use Machine Learning model to generate text and display it on the website.
Getting Started
To train model follow below steps :
Prerequisites
-
Reddit API Key
Go to https://www.reddit.com/prefs/apps/
-
Algorithmia Account
Go to https://algorithmia.com/
-
Optional:
- Wandb Account and Wandb API Key to track training
Go to https://wandb.com/
- Firebase Account to upload files while scrapping reddit
To get firebase-adminsdk.json, goto Firebase Console, click on the Gear icon besides Project Overview and select Project Settings -> Service accounts -> Generate new private key.
-
Data Sources
- Go to Data Sources folder and download the files.
Note: If you don't have a Wandb account, remove Wandb code from the scripts. And if you don't have a Firebase account, remove Firebase code from the scripts.
Installation
-
Common Steps
- Run Reddit Scraping Script.py to scrape data from Reddit.
- Run Filter Dates.py to filter dates.
- Run Csv to Story Text.py to convert csv to story text.
- Run Split Text Files.py to split text files into smaller chunks.
- Run Vocabulary details.py to get details of the vocabulary.
-
For Word Level
- Run Word_Level_Text_Generation_First_Time.ipynb to train the model for the first time.
- Run Word_Level_Text_Generation_After_First_Time.ipynb to train the model incrementally.
-
For Character Level
- Run Charcter_Level_Text_Generation.ipynb to train the model for the first time.
- Again run Charcter_Level_Text_Generation.ipynb with changes mentioned to train the model incrementally.
-
To host Model on Algorithmia
- Go to Algorithmia and create a new project.
- Upload all the files in Data Sources to your project.
- Past Algorithmia.py to Source Code on Algorithmia.
- Add Deppendency to the project.
algorithmia>=1.0.0,<2.0 tensorflow-gpu keras h5py
- Now Build and Publish the model.
-
Now follow the steps to deploy the website, on https://github.com/dhairya0907/Creepypasta-Text-Generator-Website to generate text using website interface.
Roadmap
See the open issues for a list of proposed features (and known issues).
License
Distributed under the MIT License. See LICENSE
for more information.
Contact
Dhairya Sharma - @dhairya_0907 - [email protected]
Project Github Link: https://github.com/dhairya0907/Creepypasta-Text-Generator
Project Web Application Link: https://creepypasta-demo.vercel.app