Python Web-crawler Resources

Streaming multipart/form-data parser streaming_form_data provides a Python parser for parsing multipart/form-data input chunks (the encoding used when

ProxyBroker is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them. Features F

Introduction A dead simple crawler to get books information from Douban. Pre-requesites Python 3 Install dependencies from requirements.txt (Optional)

Python Notes App. This is a small web application maked with flask-python for add notes easily and quickly. Dependencies. You can create a virtual env

QuickInfo PYPI link : https://pypi.org/project/quickinfo/ This is the library that you've all been searching for, it's built for developers and allows

Google Scholar Web Scraping This is a python script that asks for a user to input the url for a google scholar profile, and then it writes publication

Singer is an open source standard for moving data between databases, web APIs, files, queues, and just about anything else you can think of. Th

Excalibur: A web interface to extract tabular data from PDFs Excalibur is a web interface to extract tabular data from PDFs, written in Python 3! It i

FollowSpot is a comprehensive audition tracking fullstack web application for entertainment industry professionals. This app allows users to store information/media for all of their auditions while also compiling data and displaying statistics to help track progress.

Chaos Chaos is the web service which can feed Navitia with real-time disruptions. It can work together with Kirin which can feed Navitia with real-tim

stash-vr-companion This is a companion web application to connect stash to deovr. Stash is a self hosted web application to manage your porn collectio

Hey there, this is Greppo... A Python framework for building geospatial web-applications. Greppo is an open-source Python framework that makes it easy

Python REMote Interface library. Platform independent. In about 100 Kbytes, perfect for your diet.

OUO.IO LINK SHORTENER This is a simple python script that made to short links. currently ouo.io doesn't have Application Programming Interface so i de

Introduction A dead simple crawler to get books information from Douban. Pre-requesites Python 3 Install dependencies from requirements.txt (Optional)

Playwright Browser Pool This example illustrates how it's possible to use a pool of browsers to retrieve page urls in a single asynchronous process. i

apysc apysc is the Python frontend library to create HTML and js files, that has ActionScript 3 (as3)-like interface. Notes: Currently developing and

##RESTful Web API para exposição de pontos turísticos de uma região## Propor um novo ponto turístico Moderação dos pontos turísticos cadastrados Lista

Matrix Content Scanner A web service for scanning media hosted by a Matrix media repository Installation TODO Development In a virtual environment wit

Translates LinkML schemas into Datalog programs and executes them using Souffle, enabling advanced validation and inference over instance data

Discover Flask Full Stack Web Development with Flask. http://discoverflask.com Flask is a micro web framework powered by Python. Its API is fairly sma

Guitar123 version 0.8.5 Guitar tabs web app for guitar fans, powered by Python/Flask Features Guitar tabs search and browse Easy to use for end user a

Parallel TTS web demo based on Flask + Vue (Vuetify).

Financial Analysis Web App https://share.streamlit.io/mayurk1/fin-web-app-final-project/webApp.py 1. Project Description This project is a technical a

Proof of Concept GnuCash Webinterface This may one day be a something truly great. Milestones [ ] Browse accounts and view transactions [ ] Record sim

dos-attacker ❕ Atenção Não ataque sites privados. isto é illegal. 🖥️ Pré-requisitos Ultima versão do Python3. para verificar isto, é bem simples. Bas

Gypsylist gypsylist.py is a web scraper for nomadlist.com, made to avoid website restrictions. nomadlist.com is a website with a lot of information fo

Templator Simple, elegant Python based web templating (part of web.py). If you are familiar with Python, there is no new syntax to learn. This is a st

Toto Toto is a small framework intended to accelerate web service development. It is built on top of Tornado and can currently use MySQL, MongoDB, Pos

🦑 dirbruter Simples brute forcer de diretorios para web pentest. ❕ Atenção Não ataque sites privados. Isto é illegal. 🖥️ Pré-requisitos Ultima versã

Wagtail Lottie Wagtail + Lottie is a Wagtail package for playing Adobe After Effects animations exported as json with Bodymovin. Usage Export your ani

Proof of Concept GnuCash Webinterface This may one day be a something truly great. Milestones [ ] Browse accounts and view transactions [ ] Record sim

A web RTSP play platform based on websocket and tornado, websocket use blob binaryType read as ArrayBuffer

PyMDL An API for MyDramaList(MDL) based on webscraping for python. Description An API for MDL to make your life easier in retriving and working on dat

Book Search API Developer: Vladimir Vojtenko Book search Django web project that uses requests python library and openlibrary API. #requests #openlibr

A small ASGI web framework that you can make asynchronous web applications using uvicorn with using few lines of code

webScrap WebScraping first step. Authors: Paulo, Claudio M. First steps in Web Scraping. Project carried out for training in Web Scrapping. The export

Py Crypto Bot Using Binance WebSocket API to get real-time price data for cryptocurrencies. Using the TA-Lib library to calculate the RSI and execute

Webshell client A webshell client written in python. Only works well for linux for the time being. Why? Because there are too many heavy webshells. So

Viz-It Data Visualizer Web-Application If I ask you where most of the data wrangler looses their time ? It is Data Overview and EDA. Presenting "Viz-I

RecipeBook - Tandoor Python web-app (Flask) to browse Tandoor recipes on the local network. Designed for use with E-Ink screens. For a version that wo

geemap-apps Introduction This repo demostrates how to build a multi-page Earth Engine App using streamlit and geemap. You can deploy the app on variou

Short web app using Flask, tested with unittest on making massive requests, responses of the website, containerized

element_search with Selenium (Now With docstrings 😎 ) Just to mention, I'm a beginner to all this, so it it's very possible to make some mistakes The

opsec-aggressor Aggressor script that gets the latest commands from CobaltStrikes opsec page and creates an aggressor script based on tool options. Gr

Web and PDF Scraper Refactoring This repository contains the example code of the Web and PDF scraper code roast. Here are the links to the videos: Par

A platform used with frabit-server and frabit

webapp-wordlists This repository contains wordlists for each versions of common web applications and content management systems (CMS). Each version co

fastapi-jwt FastAPI native extension, easy and simple JWT auth

Story Contest Simple web site for sharing your short stories and beautiful pictures.(Cloud computing first assignment) Clouds The table below shows cl

🎶 DescargarCanciones 🎶 Buscar y descargar canciones o playlist de Spotify o YouTube automáticamente con todos los metadatos de la canciones en forma

Wappalyzer CLI tool to find Web Technologies

About Solve CTF offline disconnection problem - based on python3's small crawler, support keyword search and local map bed establishment, currently support Jianshu, xianzhi,anquanke,freebuf,seebug

cybernews A package that provides you Latest Cyber/Hacker News from website using Web-Scraping. Latest Cyber/Hacker News Using Webscraping Developed b

Crawler Rottentomatoes, Goodreads and IMDB sites crawler. Crawler written by beautifulsoup, selenium and lxml to gather books and films information an

Xnuvers007 PortInjection this is tools hacking for scan vuln in port web, happy using view/show python 3.9 solo coder (tangerang) 19 y/o installation

cliparchive Description A set of scripts to download games from the Wayback Machine's archive of Miniclip.com, and a Web frontend to play them using r

TDTV2-Direct Version 1.00.1 • A Simple Web Scraper made to Extract Download Links from Todaytvseries2.com :) How to Works?? install all dependancies v

over-the-wire-natas Natas teaches the basics of serverside web-security. Each level of natas consists of its own website located at http://natasX.nata

thebookrain Basic Form Web Development using Python, Django and CSS This is a basic project that contains two forms - borrow and donate. The form data

A web-based chat application that enables multiple users to interact with one another, in the same chat room or different ones according to their choosing.

Angular Desktop UI This is a collection for native desktop like user interface components in Angular, especially useful for Electron apps. It starts w

Introduction Asynchronous cache manager designed for horizontally scaled web applications. NOTE: Currently has implementation only for FastAPI using R

Djask is a web framework for python which stands on the top of Flask and will be as powerful as Django.

This is a Python program to Entity Linking by processing WARC files. We recognize entities from web pages and link them to a Knowledge Base(Wikidata).

Simple and easy to use python API for the COVID registration booking system of the math department @ unipd (torre archimede). This API creates an interface with the official browser, with more useful functionalities.

We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them. This module is designed to do the URL/web attestation by using the API from NUS-Phishperida-Project.

A web Scraper for CSrankings.com that scrapes University and Faculty list for a particular country To run the file: Open terminal

Comment Webpage Screenshot is a GitHub Action that helps maintainers visually review HTML file changes introduced on a Pull Request by adding comments with the screenshots of the latest HTML file changes on the Pull Request

Web scraped S&P 500 Data from Wikipedia using Pandas and performed Exploratory Data Analysis on the data. Then used Yahoo Finance to get the related stock data and displayed them in the form of charts.

Quick WAF "paranoid" Doctor Evaluation WAFPARAN01D3 The Web Application Firewall Paranoia Level Test Tool. — From alt3kx.github.io Introduction to Par

CLI Web-CAT CLI Web-CAT interface. Installation git clone https://github.com/phuang1024/cliwebcat cd cliwebcat python setup.py bdist_wheel sdist cd di

Rendering of live Jupyter notebooks with interactive widgets. Introduction Voilà turns Jupyter notebooks into standalone web applications. Unlike the

Coinbase Blog Scraper A Python web scraper to scrape latest posts from official Coinbase's Blog. IDEA It scrapes up latest blog posts from https://blo

Welcome! Open up two terminals, one for client and for server each Terminal 1 Terminal 2 Now navigate to the CW2_code directory in both like so $ cd C

Zillow-Scraper Instructions All terminal commands are highlighted. Make sure you first have python 3 installed. You can check this by running "python

A toolkit for web reconnaissance, it's fast and easy to use. File Structure httpsuite/ main.py init.py db/ db.py init.py subdomains_db directories_db

Auto-Spotify using Voice Recognition A Simple Script that will help you to Play / Change Songs with just your Voice Explore the docs » Table of Conten

This project contains the process from building a web crawler to extract the raw data of house price to create ETL pipelines using Google Could Platform services.

Simple-Port-Scanner a simple port scanner for Web/ip scanning Port 0/500 editable inside the .py file Open Cmd/Terminal Cmd Downloads Run Command: pip

pwbloom Simple web index to use bloom filter for Pwned Passwords The index.py runs a simple CGI web service checking passwords with a bloom filter for

Django REST Todo app This is a simple Todo web application built with Django (back-end) and React JS (front-end). The project enables you to systemati

yafuzz Yet another web fuzzer Usage This script can run in two modes of operation. Supplying a wordlist -W argument will initiate a multithreaded fuzz

Flask Phyton Web template A template themes for phyton flask website

Repositório contendo scripts Python que realizam a consulta de CPF e CNPJ diretamente no site da Receita Federal.

This snake helps you reconnect the Web, with RSS feeds! RSSerpent is an open-source software that create RSS feeds for websites that do not provide an

Telebirr This is a python helper package for Telebirr H5 Web payment integration helper. The use case for this interface is payment via web. Third par

Google Maps Crawler using Selenium Built as part of the Antifragile Dev Project Selenium crawler that browses Google Maps as a regular user and stores

CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training This is the official repository for the code and models of the paper CCQA: A N

Final_Project Web-server with parser, connection to DBMS and the Hugging Face. Team: Aisha Bazylzhanova(SE-2004), Arysbay Dastan(SE-2004) Installation

A Telegram Bot to convert http Links to PDF

Interactive Web App with Streamlit and Scikit-learn that applies different Classification algorithms to popular datasets Datasets Used: Iris dataset,

PyWeb 🕸️ 🐍 Current Release: 0.1 A Hobby Project 🤓 PyWeb is a small Library to generate customized static web pages using python. Aimed for new deve

crawlersuseragents This Python script can be used to check if there is any differences in responses of an application when the request comes from a se

Simple Web Application This is a simple web application using Python Flask and MySQL database. This is used in the demonstration of development of Ans

Singify Introducing Singify, the party game! Challenge your friend to who knows songs better. Play random songs from your very own Spotify playlist an

ObsPy is an open-source project dedicated to provide a Python framework for processing seismological data. It provides parsers for common file formats

Morning Messages A simple python script to send cute messages to my boyfriend. It gives him the weather and news currently. Installation git clone htt

smsir smsir is a Python library for using SMS web services www.sms.ir Installation Use the package manager pip to install smsir. pip install smsir Usa

skdue Description Skdue is a web application that makes your life easier by helping you manage your schedule. With the ability which allows you to cre