Python Web-scraping Resources

ezcv A python-based static site generator for setting up a CV/Resume site Table of Contents What does ezcv do? Features & Roadmap Why should I use ezc

Facebook Comment Downloader A small web app for downloading comments from a public facebook page post. Comment downloading from https://github.com/min

dupgee Dupgee is a mini web framework developed for micro-python(Tested on esp8266). Installation pip install dupgee Create Project dupgee create newp

Overview BisQue is a web-based platform specifically designed to provide researchers with organizational and quantitative analysis tools for up to 5D

Web Scraping images using Selenium and Python A propos de ce document This is a markdown document about Web scraping images and videos using Selenium

quiz-app Quiz Web Application made with flask and mongodb as the Databases Before you run this application, change the inside MONGODB_URI ( in config.

Face-Detection-flask-gunicorn-nginx-docker This is a simple implementation of dockerized face-detection restful-API implemented with flask, Nginx, and

mypy-django Type stubs to use the mypy static type-checker with your Django projects This project includes the PEP-484 compatible "type stubs" for Dja

Pythonic Crawling / Scraping Framework Built on Eventlet Features High Speed WebCrawler built on Eventlet. Supports relational databases engines like

Brownant Brownant is a lightweight web data extracting framework. Who uses it? At the moment, dongxi.douban.com (a.k.a. Douban Dongxi) uses Brownant i

The quick and easy way to add versatile graphical interfaces with networking capabilities to your Python programs. Give instant access to your application to whoever you want on the Internet, without having to deploy it. Works even on your Android smartphone or tablet.

Flask Apps Open-Source web apps built with automation tools, HTML parsing and boilerplated code in Flask - - Provided by AppSeed App Generator. What i

softcatala-web-dataset This repository contains Sofcatalà web site content (articles and programs descriptions). Dataset are available in the dataset

tornado-sqlalchemy Python helpers for using SQLAlchemy with Tornado. Installation $ pip install tornado-sqlalchemy In case you prefer installing from

Flask Paper Kit Open-Source Web App coded in Flask Framework - Provided by AppSeed Web App Generator. App Features: SQLite database SQLAlchemy ORM Ses

inscriptis -- HTML to text conversion library, command line client and Web service A python based HTML to text conversion library, command line client

python-jws 🚨 This is Unmaintained 🚨 This library is unmaintained and you should probably use For histo

Pomp Pomp is a screen scraping and web crawling framework. Pomp is inspired by and similar to Scrapy, but has a simpler implementation that lacks the

Auxiliator Auxiliator is telegram bot for basic web-application analysis What for? Sometimes there is no access to your main PC, where you can scan we

assignment_32558 This is a repository for a web application developed with Django, built with Crowdbotics Table of Contents Project Structure Features

Kommander A simple web-based SSH client. It supports: entering SSH login details (including private key and custom ports) and connecting user authenti

Landmark Recognition Web-App using Streamlit Watch Tutorial for this project Source Trained model landmarks_classifier_asia_V1/1 is taken from the Ten

OctoPrint OctoPrint provides a snappy web interface for controlling consumer 3D printers. It is Free Software and released under the GNU Affero Genera

Roxx-IO - The Smart Artist AI! TO DO / IDEAS Implement Web-Scraping Functionality Figure out a less annoying (and an off button for it) text to speech

Python Testing Crawler 🐍 🩺 🕷️ A crawler for automated functional testing of a web application Crawling a server-side-rendered web application is a

Find original and updated publication dates of any web page. From the command-line or within Python, all the steps needed from web page download to HTML parsing, scraping, and text analysis are included.

TBone TBone makes it easy to develop full-duplex RESTful APIs on top of your asyncio web application or webservice. It uses a nonblocking asynchronous

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. All documentation is in the "docs" directo

pastepwn - Paste-Scraping Python Framework Pastebin is a very helpful tool to store or rather share ascii encoded data online. In the world of OSINT,

XE-Ops View operational and config data from devices running Cisco IOS-XE software. NoteS The build folder is the latest build. All other files are fo

qpcr-Analyser A web-based GUI for the qpcr package that allows easy, simple and high-throughput analysis of qPCR data. As is described in more detail

HiCBrowser : A simple web browser to visualize Hi-C and other genomic tracks Fidel Ramirez, José Villaveces, Vivek Bhardwaj Installation You can insta

ESP AP Webserver demo.mp4 This is my experiment with "mobile app development" for the ESP32. The project consists of two parts, the ESP32 code and the

geospatial-streamlit Geospatial web applications developed uisng earthengine, geemap, and streamlit. App 1 - Land Surface Temperature A simple, code-f

Data Extractor Combine XPath, CSS Selectors and JSONPath for Web data extracting. Quickstarts Installation Install the stable version from PYPI. pip i

Flask-FileAlchemy Flask-FileAlchemy is a Flask extension that lets you use Markdown or YAML formatted plain-text files as the main data store for your

Amazon AWS Web Tool (AAWT) discontinued Features Cloudfront (simple) EC2 (With Charts and Prices) EC2 Reserved EBS ELB (With Charts) Obs:. only classi

⚡ Serverless Framework – Build web, mobile and IoT applications with serverless architectures using AWS Lambda, Azure Functions, Google CloudFunctions & more! –

TWINT - Twitter Intelligence Tool No authentication. No API. No limits. Twint is an advanced Twitter scraping tool written in Python that allows for s

Boto is a Python package that provides interfaces to Amazon Web Services.

Whoisss Whoisss is a website information gatharing Tool. You can cse it to collect information about website. Usage apt-get update apt-get upgrade pkg

CurriculumNet Introduction This repo contains related code and models from the ECCV 2018 CurriculumNet paper. CurriculumNet is a new training strategy

Soft Serve monitor Soft Sevre is a very nice git server. It offers a really nice TUI to browse the repositories on the server. Unfortunately, it does

docker-from-omero-poc proof-of-concept running docker container from omero web How-to Edit test_script.py so that the BaseClient is created pointing t

flask-reactize Purpose Developing a ReactJS application requires to use nodejs as back end server. What if you want to consume external APIs: how are

RapiDAST RapiDAST provides a framework for continuous, proactive and fully automated dynamic scanning against web apps/API. Its core engine is OWASP Z

NARROWVINE-REBORN ** UPDATE 21.12.01 ** As expected Google patched its ChromeCDM Whitebox exploit by Satsuoni with a force-update on the ChromeCDM. Th

Analyse des sentiments à partir des Tweets L'application est développée par Streamlit L'analyse sentimentale est effectuée sur l'ensemble de données d

shell-hack Tribute to Chinese ant sword； A Powerful terminal based webshell controller； Usage : Usage : python3 shell-hack.py --url [URL] --w

Micro-service Создание микросервиса Цель работы Познакомиться с механизмом работы современных веб-приложений и микросервисной архитектуры в процессе в

Image processing and Machine learning labs   computer vision, image processing and machine learning on the web browser or node note Fast Fourier Trans

crawler_to_visual_gmane Analyzing an EMAIL Archive from gmane and vizualizing the data using the D3 JavaScript library. This is a set of tools that al

LFI-Hunter Searches for potentially vulnerable websites to local file inclusion, throughout the web and then exploits them for LFI A script written in

Luftmensch 1.4.7 (Español) | 1.4.3 (English) Version 1.4.7 (Español) released in October 2021. Version 1.4.3 (English) released in September 2021. 🏮

PyBuster A directory busting tool for web application penetration tester, written in python. Supports custom wordlist,recursive search. Screenshots Pr

Bookings Bookings is a web application that, backed by the power of the MariaDB Connectors and the MariaDB X4 Platform, unleashes the power of smart t

grpc-web-proto-compile NOTE: This package has been superseded by romnn/proto-compile, which provides the same functionality but offers a lot more flex

slocpi-scraper Sun Life of Canada Philippines Inc Investment Funds Scraper Install dependencies pip install -r requirements.txt Usage General format:

Database_Project Prototype ideas for website: Computer Science library (Sells books, products, etc.) Code editor Graph visualizer / creator (can save

WhatsApp Bulk Message Sender - WBMS WBMS automates sending of message to multiple numbers via WhatsApp Web. Report Bug · Request Feature Love the proj

Savee - Images Downloader Project using Selenium Webdriver to download images from someone's profile on https:www.savee.it website. Usage The project

DaProfiler allows you to get emails, social medias, adresses, works and more on your target using web scraping and google dorking techniques, based in France Only. The particularity of this program is its ability to find your target's e-mail adresses.

🗃️ File.io An API wrapper for the file.io web service. Install $ pip3 install fileio or

Google Developer Profile Badge Scraper It is a Google Developer Profile Web Scraper which scrapes for specific badges in a user's Google Developer Pro

Anonymous Dark Web Tool v1.0 Features Anonymous Mode Darkweb Search Engines Check Onion Url/s Scanning Host/IP Keep eyes on v2.0 soon. Requirement Deb

Machine Learning web App This is a machine learning web application for binary classification using streamlit options this application contains 3 clas

py4jshell Simulating Log4j Remote Code Execution (RCE) CVE-2021-44228 vulnerability in a flask web server using python's logging library with custom f

How useful is the aswer? A Streamlit web-app for a data-science project that aims to evaluate if the answer to a question is helpful. If you want to l

ML-Based-Dog-Breed-Identifier This is a Django Based Web Site To Identify the Breed of which your DOG belogs All You Need To Do is to Follow These Ste

PyYoutubemanager Explore the docs » View Demo · Report Bug · Request Feature About The Project PyYotubemanager is full web automated bot capable of Ge

Wonkey Programming Language Wonkey is an open source programming language for the creation of cross-platform video games, highly inspired by the “Blit

Robert A simple Python web searcher library with different search engines. Install pip install roberthelper Usage from robert import GoogleSearcher

Telegram Link Wayback Bot. This bot archives a web page thrown at itself with wayback Machine (Archive.org).

Fsocket Extend frappe's websocket server using socket.io and redis Installation Use frappe bench to add fsocket in your project $ bench get-app https:

Kalidoface 3D - Face and Full-Body tracking for Vtubing on the web! A sequal to Kalidoface which supports Live2D avatars, Kalidoface 3D is a web app t

FastWSGI - An ultra fast WSGI server for Python 3

Log4jake Log4jake works by spidering a web application for GET/POST requests. It will then automatically execute the GET/POST requests, filling any di

Table of Contents Features Install Using PIP Stable version Development version Using Docker Stable version Development version Using HomeBrew Stable

society-email-scrape Site Live at https://kcsoc.github.io/society-email-scrape/ How to automatically generate new data Go to unis.yml Add your uni Cre

Neural Style Transfer Web App - [FastAPI + streamlit + Docker] NST - application based on the Perceptual Losses for Real-Time Style Transfer and Super

I-P-L-Team-Project This "I P L Team Project" is developed by Prasanta Kumar Mohanty using Python with Django web framework, HTML & CSS. Screenshots HO

알고리즘 스터디 🔥 부스트캠프 웹모바일 6기 iOS 10조의 알고리즘 스터디 입니다. 개인적인 사정 등으로 S034, S055만 참가하였습니다. 스터디 목적 상진: 코테 합격 + 부캠끝나고 아침에 일어나기 위해 필요한 사이클 기완: 꾸준하게 자리에 앉아 공부하기 +

Apicell You can use this api to search in google, bing, pypi and subscene and get results Method : POST Parameter : query Example import request url =

mypy-playground The mypy playground provides Web UI to run mypy in the sandbox: Features Web UI and sandbox for running mypy eas

JustPy Docs and Tutorials Introduction JustPy is an object-oriented, component based, high-level Python Web Framework that requires no front-en

javaEeAccessControlCheck Check broken access control exists in the Java web application. 检查 Java Web 应用程序中是否存在访问控制绕过问题。 使用 python3 javaEeAccessControl

Playlist Converter Convert a text file of songs to a playlist on your Spotify account. Create your playlists faster instead of manually searching for

lambdarest Python routing mini-framework for AWS Lambda with optional JSON-schema validation. ⚠️ A user study is currently happening here, and your op

H2O Wave Realtime Web Apps and Dashboards for Python and R New! R Language API Build and control Wave dashboards using R! New! Easily integrate AI/ML

Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data anal

A package for building highly interactive user interfaces in pure Python inspired by ReactJS.

pyramid_services The core of a service layer that integrates with the Pyramid Web Framework. pyramid_services defines a pattern and helper methods for

RDFLib RDFLib is a pure Python package for working with RDF. RDFLib contains most things you need to work with RDF, including: parsers and serializers

Klein, a Web Micro-Framework Klein is a micro-framework for developing production-ready web services with Python. It is 'micro' in that it has an incr

This is a webscraper for a specific website. It is tuned to extract the headlines of that website. With some little adjustments the webscraper is able to extract any part of the website.

Annex Bubt Scraping Script I think this is the first public repository that provides free annex-BUBT, BUBT-Soft, and BUBT website scraping API script

t5-japanese Codes to pre-train T5 (Text-to-Text Transfer Transformer) models pre-trained on Japanese web texts. The following is a list of models that

Package Description Status PyNHD Navigate and subset NHDPlus (MR and HR) using web services Py3DEP Access topographic data through National Map's 3DEP

Ballcone Ballcone is a fast and lightweight server-side Web analytics solution. It requires no JavaScript on your website. Screenshots Design Goals Si

Streaming multipart/form-data parser streaming_form_data provides a Python parser for parsing multipart/form-data input chunks (the encoding used when