Download all posts and comments in a subreddit

Related tags

Downloader subreddit_downloader
Overview

subreddit downloader

This subreddit downloader downloads all posts and comments in a subreddit

For a tutorial to use this program please follow this medium article - https://medium.com/@guneet.android/download-subreddit-data-for-nlp-projects-c8fa85c0df87

This script makes use of the pushshift API ( https://github.com/pushshift/api ) to download the entire dump of posts and comments in a specific subreddit

Simply replace the subreddit name in the fourth line of code and run the script

It will download two files one for all the posts and one for all the comments

Patched UTF-8 encoding*

Please note pushshift has rate limitations -

repeated 429 error codes will result in blocklisting of IP

If your data format is in single quotes

Please use this JSON fixer by @mbrzusto

https://gist.github.com/mbrzusto/23fe728966247f25f3ec

You might also like...
Download all games from a public Itch.io Game Jam

Itch Jam Downloader Downloads all games from a public Itch.io Game Jam. What you'll need: Python 3.8+ pip install -r requirements.txt For site mirrori

Dragoon Aethis 19 Dec 7, 2022
👻🟡 Download all Snapchat video & photo memories from a data export.
👻🟡 Download all Snapchat video & photo memories from a data export.

Snapchat "Memories" Fetcher In compliance with the California Consumer Privacy Act of 2018 (“CCPA”), businesses which collect and store user data must

Todd Birchard 18 Dec 26, 2022
FireDM is a python open source (Internet Download Manager) with multi-connections, high speed engine, it downloads general files and videos from youtube and tons of other streaming websites .
FireDM is a python open source (Internet Download Manager) with multi-connections, high speed engine, it downloads general files and videos from youtube and tons of other streaming websites .

python open source (Internet Download Manager) with multi-connections, high speed engine, based on python, LibCurl, and youtube_dl https://github.com/firedm/FireDM

null 1.6k Apr 12, 2022
Using Youtube downloader is the fast and easy way to download and save any YouTube video.
Using Youtube downloader is the fast and easy way to download and save any YouTube video.

Youtube video downloader using Django Using Django as a backend along with pytube module to create Youtbue Video Downloader. https://yt-videos-downloa

Suman Raj Khanal 10 Jun 18, 2022
Advance Image Downloader/Extractor (Job) is a Python-Flask web-based app, which will help the user download the any kind of Images at any date and time over the internet. These images will get downloaded as a job and then let user know that the images have been downloaded by sending them a link over an email.
Advance Image Downloader/Extractor (Job) is a Python-Flask web-based app, which will help the user download the any kind of Images at any date and time over the internet. These images will get downloaded as a job and then let user know that the images have been downloaded by sending them a link over an email.

Advance Image Downloader/Extractor(Job) Advance Image Downloader/Extractor (Job) is a Python-Flask web-based app, which will help the user download th

Shreyas 13 Aug 27, 2022
Download and save Bing wallpapers and set as background for GNOME desktop

Save Bing wallpapers and set as background for GNOME desktop This script downloads the Bing wallpaper and sets it in the background of your gnome desk

manikamran 2 Nov 6, 2021
Command-line program to download videos from YouTube.com and other video sites

youtube-dl - download videos from youtube.com or other video platforms

youtube-dl 116.4k Jan 7, 2023
The free and open-source Download Manager written in pure Python
The free and open-source Download Manager written in pure Python

The free and open-source Download Manager written in pure Python

pyLoad 2.7k Dec 31, 2022
A Udemy downloader that can download DRM protected videos and non-DRM protected videos.
A Udemy downloader that can download DRM protected videos and non-DRM protected videos.

Udemy Downloader with DRM support NOTE This program is WIP, the code is provided as-is and i am not held resposible for any legal repercussions result

Puyodead1 468 Dec 29, 2022
Comments
  • utf-8 encoding should be default for the open() statement

    utf-8 encoding should be default for the open() statement

    I found this script with google, it's insane that it's only been published yesterday. Anyway, to make it work for me, I had to add this to the open() statement: encoding="utf-8" So that would make: with open(subreddit + '_' + thing + '_' + str(int(time.time())) + '.txt', 'w', encoding="utf-8") as f:

    other than that, thanks for the script, it's great!

    opened by Silver-Volt4 2
Owner
Guneet
Guneet
GitHub
YT-Spammer-Purge - Allows you easily scan for and delete scam comments using several methods

YouTube Spammer Purge What Is This? - Allows you to filter and search for spamme

null 4.3k Dec 31, 2022
A web app for downloading Facebook comments as a csv file

Facebook Comment Downloader A small web app for downloading comments from a public facebook page post. Comment downloading from https://github.com/min

WSDOT 23 Jan 4, 2023
Download all your URI Online Judge source codes and upload to GitHub with simple steps.

URI-Code-Downloader Download all your URI Online Judge source codes and upload to GitHub with simple steps. Prerequisites Python 3.x Installing Downlo

Luan Simões 9 Mar 23, 2022
You Can download any video/image in all social medias very easy and High Speed.

All-Downloader You Can download any video/image in all social medias very easy and High Speed. also you can easily download videos from web browsers s

Razor Kenway 14 Dec 16, 2022
A tool written in Python to download all Snapmaps content from a specific location.

snapmap-archiver A tool written in Python to download all Snapmaps content from a specific location.

null 46 Dec 9, 2022
This is a python based web scraping bot for windows to download all ACCEPTED submissions of any user on Codeforces

CODEFORCES DOWNLOADER This is a python based web scraping bot for windows to download all ACCEPTED submissions of any user on Codeforces Requirements

Mohak 6 Dec 29, 2022
A Python script that allows you to download all of an anime's episodes at once.

BitAnime A Python script that allows you to download all of an anime's episodes at once. · Download executable version · About BitAnime BitAnime is a

sh1nobu 17 Aug 10, 2022
Python script to download all images/webms of a 4chan thread

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

Micha Fink 208 Jan 4, 2023
Download all games from a public Itch.io Game Jam

Itch Jam Downloader Downloads all games from a public Itch.io Game Jam. What you'll need: Python 3.8+ pip install -r requirements.txt For site mirrori

Dragoon Aethis 19 Dec 7, 2022
python code used to download all images contained in a facebook uid , the uid can be profile,group,fanpage

python code used to download all images contained in a facebook uid , the uid can be profile,group,fanpage

VVHai 2 Dec 21, 2021