Python script for crawling ResearchGate.net papers
This code start crawling process by urls in start.txt and give paper details in crawled.json.
First install Python. Then install these libraries:
pip install selenium
pip install webdriver-manager
MAX_FETCH_COUNT: How many pages you want to crawl?
MAX_CACHED_NUM: We renew crawled.json after crawling each MAX_CACHED_NUM papers.
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
🐛 MASS DBLTEK EXPLOIT CHECKER USING SHODAN 🕸 Python script who crawl first shodan page and check DBLTEK vulnerability
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
Aliexpress to telegram post Python script that reads Aliexpress offers urls from a Excel filename (.csv) and post then in a Telegram channel using a b
This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers. The script reads in the d
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
#Amazon-web-scraper This is a python program, which use scrapy python framework to crawl all pages of the product and scrap products data. This progra
Cola: high-level distributed crawling framework Overview Cola is a high-level distributed crawling framework, used to crawl pages and extract structur
Cola: high-level distributed crawling framework Overview Cola is a high-level distributed crawling framework, used to crawl pages and extract structur
Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp. Requirements Python3.5+ Installation pip install gain pip install uvloo
Requests and Responses¶ Scrapy uses Request and Response objects for crawling web sites. Typically, Request objects are generated in the spiders and p
PaperRobot PaperRobot 是一个论文抓取工具，可以快速批量下载大量论文，方便后期进行持续的论文管理与学习。 PaperRobot通过多个接口抓取论文，目前抓取成功率维持在90%以上。通过配置Config文件，可以抓取任意计算机领域相关会议的论文。 Installation Down
paper finder Find papers by keywords and venues. Then download it automatically. How to use this? Search CLI python search.py -k "knowledge tracing,kn
Scroodle Selenium script to scrape scrambles from csTimer.net csTimer runs locally in your browser, so this doesn't strain the servers any more than i
This repo contains the source code for crawler and crawled data of cars specifications from autodata. The data has roughly 45k cars
crawlersuseragents This Python script can be used to check if there is any differences in responses of an application when the request comes from a se
0 Dec 15, 2021
4 Jan 9, 2022
10 Aug 18, 2022
1 Jan 2, 2022
68 Oct 8, 2022
6 Dec 6, 2022
71 Oct 4, 2022
11 Dec 11, 2022
4 Nov 9, 2021
1 Dec 26, 2021
1.5k Jan 4, 2023
2k Jan 5, 2023
1 Nov 3, 2021
47 Nov 23, 2022
2 Dec 15, 2022
1 Oct 29, 2021
5 Nov 22, 2022
13 Dec 27, 2022