339 Repositories
Python gelbooru-scraping-utility Libraries
Cross-platform CLI tool to generate your Github profile's stats and summary.
ghs Cross-platform CLI tool to generate your Github profile's stats and summary. Preview Hop on to examples for other usecases. Jump to: Installation
A comand-line utility for taking automated screenshots of websites
shot-scraper A comand-line utility for taking automated screenshots of websites For background on this project see shot-scraper: automated screenshots
Python utility to generate filesystem content for Obsidian.
Security Vault Generator Quickly parse, format, and output common frameworks/content for Obsidian.md. There is a strong focus on MITRE ATT&CK because
Face recognition system using MTCNN, FACENET, SVM and FAST API to track participants of Big Brother Brasil in real time.
BBB Face Recognizer Face recognition system using MTCNN, FACENET, SVM and FAST API to track participants of Big Brother Brasil in real time. Instalati
An introduction to free, automated web scraping with GitHub’s powerful new Actions framework.
An introduction to free, automated web scraping with GitHub’s powerful new Actions framework Published at palewi.re/docs/first-github-scraper/ Contrib
Materials to reproduce our findings in our stories, "Amazon Puts Its Own 'Brands' First Above Better-Rated Products" and "When Amazon Takes the Buy Box, it Doesn’t Give it up"
Amazon Brands and Exclusives This repository contains code to reproduce the findings featured in our story "Amazon Puts Its Own 'Brands' First Above B
Dude is a very simple framework for writing web scrapers using Python decorators
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax.
A utility that allows you to use DI in fastapi without Depends()
fastapi-better-di What is this ? fastapi-better-di is a utility that allows you to use DI in fastapi without Depends() Installation pip install fastap
Optimizes image files by converting them to webp while also updating all references.
About Optimizes images by (re-)saving them as webp. For every file it replaced it automatically updates all references. Works on single files as well
Fast TikTok NO Watermark Video Downloader (username or url)
💎 TD [ TikDown v4 ] Star ⭐ if you want more Discord Server * discord.gg/onlp | Waxor#9999 Why not open source anymore ? * BECAUSE PEOPLE SKID, STEA
An command-line utility that schedules your exams preparation routines
studyplan A tiny utility that schedules your exams preparation routines. You only need to specify the tasks and the deadline. App will output a iCal f
Code for the Open Data Day 2022 publicbodies.org Nepal data scraping activities.
Open Data Day Publicbodies.org Nepal We've gathered on Saturday, 5th March 2022 with Open Knowledge Nepal in order to try and automate the collection
Used to record WKU's utility bills on a regular basis.
WKU水电费小助手 一个用于定期记录WKU水电费的脚本 Looking for English Readme? 背景 由于WKU校园内的水电账单系统时常存在扣费延迟的现象,而补扣的费用缺乏令人信服的证明。不少学生为费用摸不着头脑,但也没有申诉的依据。为了更好地掌握水电费使用情况,留下一手证据,我开源
SkyScrapers: A collection of variety of Scraping Apps
SkyScrapers Collection of variety of Web Scraping Apps The web-scrapers involved
Visyerres sgdf woob - Modules Woob pour l'intranet et autres sites Scouts et Guides de France
Vis'Yerres SGDF - Modules Woob Vous avez le sentiment que l'intranet des Scouts
Scrapy-soccer-games - Scraping information about soccer games from a few websites
scrapy-soccer-games Esse projeto tem por finalidade pegar informação de tabela d
A training task for web scraping using python multithreading and a real-time-updated list of available proxy servers.
Parallel web scraping The project is a training task for web scraping using python multithreading and a real-time-updated list of available proxy serv
This utility synchronises spelling dictionaries from various tools with each other.
This utility synchronises spelling dictionaries from various tools with each other. This way the words that have been trained on MS Office are also correctly checked in vim or Firefox. And vice versa of course.
A Python based command line ARP Spoofer utility, which takes input as arguments for the exact target IP and gateway IP for which you wish to Spoof ARP request
A Python based command line ARP Spoofer utility, which takes input as arguments for the exact target IP and gateway IP for which you wish to Spoof ARP request
A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers.
Dying Light 2 PAKFile Utility A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers. This tool aims to make PAKFile (.pak files) modding a
This is a python based command line Network Scanner utility, which input as an argument for the exact IP address or the relative IP Address range you wish to do the Network Scan for and returns all the available IP addresses with their MAC addresses on your current Network.
This is a python based command line Network Scanner utility, which input as an argument for the exact IP address or the relative IP Address range you wish to do the Network Scan for and returns all the available IP addresses with their MAC addresses on your current Network.
Python wrapper and CLI utility to render LaTeX markup and equations as SVG using dvisvgm and svgo.
latex2svg Python wrapper and CLI utility to render LaTeX markup and equations as SVG using dvisvgm and svgo. Based on the original work by Tino Wagner
Scraping and visualising India's real-time COVID-19 data from the MOHFW dataset.
COVID19-WEB-SCRAPER Open Source Tech Lab - Project [SEMESTER IV] OSTL Assignments OSTL Assignments - 1 OSTL Assignments - 2 Project COVID19 India Data
Helperpod - A CLI tool to run a Kubernetes utility pod with pre-installed tools that can be used for debugging/testing purposes inside a Kubernetes cluster
Helperpod is a CLI tool to run a Kubernetes utility pod with pre-installed tools that can be used for debugging/testing purposes inside a Kubernetes cluster.
Simple library for exploring/scraping the web or testing a website you’re developing
Robox is a simple library with a clean interface for exploring/scraping the web or testing a website you’re developing. Robox can fetch a page, click on links and buttons, and fill out and submit forms.
Track your favorite minecraft servers' (too bee too tee) queue at any time at your own convenience!
2b2t Queue Monitor Track your favorite minecraft servers' (too bee too tee) queue at any time at your own convenience! Compiled Usage go to https://gi
slipit is a command line utility for creating archives with path traversal elements.
slipit is a command line utility for creating archives with path traversal elements. It is basically a successor of the famous evilarc utility with an extended feature set and improved base functionality.
A Python command-line utility for validating that the outputs of a given Declarative Form Azure Portal UI JSON template map to the input parameters of a given ARM Deployment Template JSON template
A Python command-line utility for validating that the outputs of a given Declarative Form Azure Portal UI JSON template map to the input parameters of a given ARM Deployment Template JSON template
A voice control utility for Spotify
Spotify Voice Control A voice control utility for Spotify · Report Bug · Request
EVE-NG tools, A Utility to make operations with EVE-NG more friendly.
EVE-NG tools, A Utility to make operations with EVE-NG more friendly. Also it support different snapshot operations with same style as Libvirt/KVM
metovlogs is a very simple logging library
metovlogs is a very simple logging library. Setup is one line, then you can use it as a drop-in print replacement. Sane and useful log format out of the box. Best for small or early projects.
split-manga-pages: a command line utility written in Python that converts your double-page layout manga to single-page layout.
split-manga-pages split-manga-pages is a command line utility written in Python that converts your double-page layout manga (or any images in double p
Mednlp - Medical natural language parsing and utility library
Medical natural language parsing and utility library A natural language medical
A simple app to scrap data from Twitter.
Twitter-Scraping-App A simple app to scrap data from Twitter. Available Features Search query. Select number of data you want to fetch from twitter. C
An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy, Zooqle, Kickass, Bitsearch, and MagnetDL
An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy, Zooqle, Kickass, Bitsearch, and MagnetDL
Utility to add/remove licenses to/from source files
Utility to add/remove licenses to/from source files. Supports processing any combination of globs, files, and directories (recurse). Pruning options allow skipping non-licensing files.
Fetch fund data from avanza.se using Python and some web scraping with bs4
Py(A)vanza Fetch fund data from avanza.se using Python and some web scraping with bs4. The default way is to display the data in the terminal, apply -
Amazon web scraping using Scrapy Framework
Amazon-web-scraping-using-Scrapy-Framework Scrapy Scrapy is an application framework for crawling web sites and extracting structured data which can b
Basic-html-scraper - A complete how to of web scraping with Python for beginners
basic-html-scraper Code from YT Video This video includes a complete how to of w
This was supposed to be a web scraping project, but somehow I've turned it into a spamming project
Introduction This was supposed to be a web scraping project, but somehow I've turned it into a spamming project.
Linkedin webscraping - Linkedin web scraping with python
linkedin_webscraping This is the first step of a full project called "LinkedIn J
Truetool - A TrueCharts automatic and bulk update utility
truetool A easy tool for frequently used TrueNAS SCALE CLI utilities. Previously
Explore scraping with BeautifulSoup!
beautifulsoup-scrape Explore scraping with BeautifulSoup! Part One: Start from Shakespeare As my professor is a poet (yes, and he teaches me data and
A work in progress box containing various Python utilities
python-wipbox A set of modern Python libraries under development to simplify the execution of reusable routines by different projects. Table of Conten
Scraper pour les offres de stage Tesla et les notes sur Oasis (Polytech Paris-Saclay) sous forme de bot Discord
Scraper pour les offres de stage Tesla et les notes sur Oasis (Polytech Paris-Saclay) sous forme de bot Discord
Booky - A command line utility for bookmarking files on your terminal!
Booky A command line utility for bookmarking files for quick access With it you can: Bookmark and delete your (aliases of) files at demand Launch them
This is a Web scraping project using BeautifulSoup and Python to scrape basic information of all the Test matches played till Jan 2022.
Scraping-test-matches-data This is a Web scraping project using BeautifulSoup and Python to scrape basic information of all the Test matches played ti
Simple encryption/decryption utility using Pycryptodome module. Working with AES and RSA algorithms.
EncypherUtil Simple encryption/decryption utility using PyCryptodome module. Working with AES and RSA algorithms. THIS UTILITY IS NOT LICENSED AS CRYP
Desktop utility to download images/videos/music/text from various websites, and more
Desktop utility to download images/videos/music/text from various websites, and more
A TrueCharts automatic and bulk update utility
trueupdate A TrueCharts automatic and bulk update utility How to install run pip install trueupdate Please be aware you will need to reinstall after e
An application that on a given url, crowls a web page and gets all words, sorts and counts them.
Web-Scrapping-1 An application that on a given url, crowls a web page and gets all words, sorts and counts them. Installation Using the package manage
Cl datasets - PyTorch image dataloaders and utility functions to load datasets for supervised continual learning
Continual learning datasets Introduction This repository contains PyTorch image
Borderless-Window-Utility - Modifies window style to force most applications into a borderless windowed mode
Borderless-Window-Utility Modifies window style to force most applications into
Web Scraping Instagram photos with Selenium by only using a hashtag.
Web-Scraping-Instagram This project is used to automatically obtain images by web scraping Instagram with Selenium in Python. The required input will
Video Games Web Scraper is a project that crawls websites and APIs and extracts video game related data from their pages.
Video Games Web Scraper Video Games Web Scraper is a project that crawls websites and APIs and extracts video game related data from their pages. This
A Very simple free proxy list scraper.
Scrappp A Very simple free proxy list scraper, made in python The tool scrape proxy from diffrent sites and api's. Screenshots About the script !!! RE
A supercharged Git/GitHub command line interface (CLI)
A supercharged Git/GitHub command line interface (CLI).
Scraping Top Repositories for Topics on GitHub,
0.-Webscrapping-using-python Scraping Top Repositories for Topics on GitHub, Web scraping is the process of extracting and parsing data from websites
A ULauncher/Albert extension that supports currency, units and date time conversion, as well as a calculator that supports complex numbers and functions.
Ulauncher/Albert Calculate Anything Ulauncher/Albert Calculate Anything is an extension for Ulauncher and Albert to calculate things like currency, ti
General Assembly's 2015 Data Science course in Washington, DC
DAT8 Course Repository Course materials for General Assembly's Data Science course in Washington, DC (8/18/15 - 10/29/15). Instructor: Kevin Markham (
AutoKey, a desktop automation utility for Linux and X11.
AutoKey Contents About Installation Zero-installation Method Documentation Support Bug reports and Pull Requests Changelog License About AutoKey, a de
Web-scraping - Program that scrapes a website for a collection of quotes, picks one at random and displays it
web-scraping Program that scrapes a website for a collection of quotes, picks on
Python utility library for compositing PDF documents with reportlab.
pdfdoc-py Python utility library for compositing PDF documents with reportlab. Installation The pdfdoc-py package can be installed directly from the s
Generating synthetic mobility data for a realistic population with RNNs to improve utility and privacy
lbs-data Motivation Location data is collected from the public by private firms via mobile devices. Can this data also be used to serve the public goo
NFT-Image-Generator - Utility to generate a large collection of unique images
NFT-Image-Generator Utility for creating a generative art collection from suppli
Web Scraping COVID 19 Meta Portal with Python
Web-Scraping-COVID-19-Meta-Portal-with-Python - Requests API and Beautiful Soup to scrape real-time COVID statistics from worldometer website and perform data cleaning and visual analysis in Jupyter notebook.
Web-scraping - A bot using Python with BeautifulSoup that scraps IRS website by form number and returns the results as json
Web-scraping - A bot using Python with BeautifulSoup that scraps IRS website (prior form publication) by form number and returns the results as json. It provides the option to download pdfs over a range of years.
A web-based analysis toolkit for the System Usability Scale providing calculation, plotting, interpretation and contextualization utility
System Usability Scale Analysis Toolkit The System Usability Scale (SUS) Analysis Toolkit is a web-based python application that provides a compilatio
Dictionary - Application focused on word search through web scraping
Dictionary - Application focused on word search through web scraping, in addition to other functions such as dictation, spell and conjugation of syllables.
Tinybar - A simple MacOS menu bar utility to track price dynamics for assets on TinyMan.org
📃 About A simple MacOS menu bar app to display current coins from most popular
Skype export archive to text converter for python
Skype export archive to text converter This software utility extracts chat logs
TinyBar - Tiny MacOS menu bar utility to track price dynamics for assets on TinyMan.org
📃 About A simple MacOS menu bar app to display current coins from most popular Liquidity Pools on TinyMan.org
Poolbooru gelscraper - a simple python script for scraping images off gelbooru pools.
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
A python script to extract answers to any question on Quora (Quora+ included)
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
A cross-platform python based utility to download courses from udemy for personal offline use.
udemy-dl A cross-platform python based utility to download courses from udemy for personal offline use. Warning Udemy has started to encrypt many of t
Ntfy - 🖥️📱🔔 A utility for sending notifications, on demand and when commands finish.
About ntfy ntfy brings notification to your shell. It can automatically provide desktop notifications when long running commands finish or it can send
Browse Hacker News like a haxor: A Hacker News command line interface (CLI).
haxor-news Coworker who sees me looking at something in a browser: "Glad you're not busy; I need you to do this, this, this..." Coworker who sees me s
Autosub - Command-line utility for auto-generating subtitles for any video file
Auto-generated subtitles for any video Autosub is a utility for automatic speech recognition and subtitle generation. It takes a video or an a
Bigdata - This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster
Scrapy Cluster This Scrapy project uses Redis and Kafka to create a distributed
FileGuard - File crypter and packing utility
FILEGUARD FILEGUARD is a file crypter and packing utility. This project was orig
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
PeGuard - Windows PE crypter and packing utility
PEGUARD PEGUARD is a file crypter and packing utility. This project was original
Web Scraping com Python - Raspando Vagas para Programadores
Web Scraping com Python - Raspando Vagas para Programadores Sobre o Projeto Web
Utility for downloading works from AO3 (Archive Of Our Own)
froyo A small graphical application for batch downloading works from Archive Of Our Own (AO3). Curate a fic repo of your own today :) Features Batch d
Utility to find games owned by all (or at least some) of the passed players.
SteamCommonGameFinder Utility to find games that are owned by all (or at least some) of the players you pass into this programm. You can already find
Web Scraping images using Selenium and Python
Web Scraping images using Selenium and Python A propos de ce document This is a markdown document about Web scraping images and videos using Selenium
Pythonic Crawling / Scraping Framework based on Non Blocking I/O operations.
Pythonic Crawling / Scraping Framework Built on Eventlet Features High Speed WebCrawler built on Eventlet. Supports relational databases engines like
Utility for downloading works from AO3 (Archive Of Our Own)
ao3d video preview A small graphical utility for batch downloading works from AO3 (Archive Of Our Own) Features Batch downloading works to supported f
SubGrab is a utility that allows you to automate subtitles downloading for your media files.
SubGrab - Command-line Subtitles Downloader: A utility which provides an ease for automating media i.e., Movies, TV-Series subtitle scraping from mult
Screen scraping and web crawling framework
Pomp Pomp is a screen scraping and web crawling framework. Pomp is inspired by and similar to Scrapy, but has a simpler implementation that lacks the
Fast and robust date extraction from web pages, with Python or on the command-line
Find original and updated publication dates of any web page. From the command-line or within Python, all the steps needed from web page download to HTML parsing, scraping, and text analysis are included.
Utility code for use with PyXLL
pyxll-utils There is no need to use this package as of PyXLL 5. All features from this package are now provided by PyXLL. If you were using this packa
Python framework to scrape Pastebin pastes and analyze them
pastepwn - Paste-Scraping Python Framework Pastebin is a very helpful tool to store or rather share ascii encoded data online. In the world of OSINT,
Utility for Google Text-To-Speech batch audio files generator. Ideal for prompt files creation with Google voices for application in offline IVRs
Google Text-To-Speech Batch Prompt File Maker Are you in the need of IVR prompts, but you have no voice actors? Let Google talk your prompts like a pr
Audio pitch-shifting & re-sampling utility, based on the EMU SP-1200
Pitcher.py Free & OS emulation of the SP-12 & SP-1200 signal chain (now with GUI) Pitch shift / bitcrush / resample audio files Written and tested in
Drover is a command-line utility for deploying Python packages to Lambda functions.
drover drover: a command-line utility for deploying Python packages to Lambda functions. Background This utility aims to provide a simple, repeatable,
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
TWINT - Twitter Intelligence Tool No authentication. No API. No limits. Twint is an advanced Twitter scraping tool written in Python that allows for s
to-requirements.txt allows to automatically add and delete modules to requirements.txt installing them using pip.
to-requirements.txt | Automatically update requirements.txt to-requirements.txt allows to automatically add and delete modules to requirements.txt ins
A morse code encoder and decoder utility.
morsedecode A morse code encoder and decoder utility. Installation Install it via pip: pip install morsedecode Alternatively, you can use pipx to run