A basic API to scrape Craigslist.

Overview

CLAPI

A basic API to scrape Craigslist.

Most useful for viewing posts across a broad geographic area or for viewing posts within a specific timeframe.


Requirements:

  • bs4 (BeautifulSoup)
  • shutil
  • requests
  • datetime
  • PyQt5
  • subprocess

Note: All of these packages should be available from standard distributions, such as Anaconda.


Typical Use Case:


from CLAPI import CraigsList
cities = []
for state in ['AL', 'AK', 'AZ', 'AR']:
    cities += CraigsList.GetCitiesByState(state)
hours = float(input('Posts in the last x hours >> '))/24
query = input('Query >> ')
for city in cities:
    print('Parsing %s...' % city)
    cl = CraigsList(city, query, CraigsList.SORT_RELEVANT, lookback=hours)
    posts += cl.posts
CraigsList.OpenViewer(posts, maxImgs=3)

The above example scrapes the posts during the lookback period for every city with a Craiglist in the specified states. These posts are presented to the user in a simple PyQt5 GUI for rapid browsing. The user can quickly open the associated post webpage or post location via buttons on the GUI.

Note: if you use a browser other than chrome, you will want to modify the subprocess call in the MainWindowHandlers.py file such that you call the appropriate browser.

Sample Viewer

Be aware, this program will create a temporary directory within your current working directory, called 'tmp' in which the Craigslist thumbnail images are downloaded. When the program exits without errors, this temporary directory will be deleted.

You might also like...
This is a story bot, that will scrape stories from r/stories subreddit and convert it into an Audio File.

Introduction This is a story bot, that will scrape stories from r/stories subreddit and convert it into an Audio File. Installation pip install -r req

Python script to scrape users/id/badges/creation-date from a Discord Server Memberlist
Python script to scrape users/id/badges/creation-date from a Discord Server Memberlist

Discord Server Badges Scraper - Credits to bytixo he made this Custom Discord Vanity Creator How To Use Install discum 1.2 Open a terminal and execute

A quick-and-dirty script to scrape the daily menu of Leipzig University Mensa and send it to a telegram channel.
A quick-and-dirty script to scrape the daily menu of Leipzig University Mensa and send it to a telegram channel.

Feed me Mensa UL A quick-and-dirty script to scrape the daily menu of Leipzig University Mensa and send it to a telegram channel. For food and cat lov

A Discord bot to scrape textfiles from messages and put them to Hastebin

A Discord bot to scrape textfiles from messages and put them to Hastebin. Intended to use on support servers to help users read textfiles on mobile.

scrape tiktok/douyin video list from specific user or keyword

get-tiktok-user-video-list scrape tiktok/douyin video list from specific user or keyword 以**https://www.douyin.com/user/MS4wLjABAAAAUpIowEL3ygUAahQB47

Basic Python3 request wrapper for the PancakeSwap API

🐍 Python Pancakes 🥞 A simple request wrapper for the Pancake-Swap API. Installation Install package # Using pip $ pip install pythonpancakes # Or f

A basic Ubisoft API wrapper created in python.

UbisoftAPI A basic Ubisoft API wrapper created in python. I will be updating this with more endpoints as time goes on. Please note that this is my fir

A basic Ubisoft API wrapper created in python.

UbisoftAPI A basic Ubisoft API wrapper created in python. I will be updating this with more endpoints as time goes on. Please note that this is my fir

Exchange indicators & Basic functions for Binance API.
Exchange indicators & Basic functions for Binance API.

binance-ema Exchange indicators & Basic functions for Binance API. This python library has been written to calculate SMA, EMA, MACD etc. functions wit

Owner
null
Simple Craigslist wrapper

python-craigslist A simple Craigslist wrapper. License: MIT-Zero. Disclaimer I don't work for or have any affiliation with Craigslist. This module was

Julio M. Alegria 370 Dec 22, 2022
API which uses discord+mojang api to scrape NameMC searches/droptime/dropping status of minecraft names, and texture links

API which uses discord+mojang api to scrape NameMC searches/droptime/dropping status of minecraft names, and texture links

null 2 Dec 22, 2021
Scrape the Twitter Frontend API without authentication.

Twitter Scraper ???? Read Korean Version Twitter's API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has

Buğra İşgüzar 3.4k Jan 8, 2023
wyscoutapi is an extremely basic API client for the Wyscout API (v2 & v3) for Python

wyscoutapi wyscoutapi is an extremely basic API client for the Wyscout API (v2 & v3). Usage Install with pip install wyscoutapi. To connect to the Wys

Ben Torvaney 11 Nov 22, 2022
Scrape Twitter for Tweets

Backers Thank you to all our backers! ?? [Become a backer] Sponsors Support this project by becoming a sponsor. Your logo will show up here with a lin

Ahmet Taspinar 2.2k Jan 2, 2023
Automatically scrape all of your artifacts in Genshin Impact.

Genshin Artifact Scraper Automatically scrape all of your artifacts in Genshin Impact. Features: Simple recalibration (2 steps). GUI to select OCR reg

null 21 Dec 17, 2022
A Tool to scrape URLs for a given domain from wayback machine, Commoncrawl and OTX Alienvault

Mr_URL Mr.URL fetches known URLs for a given domain from Wayback Machine, Commoncrawl and OTX Alienvault. It also finds old versions of any given URL

Stinger 9 Sep 5, 2022
Instagram bot that upload images for you which scrape posts from 9gag meme website or other Instagram users , which is 24/7 Automated Runnable.

Autonicgram Automates your Instagram posts by taking images from sites like 9gag or other Instagram accounts and posting it onto your page. Features A

Mastermind 20 Sep 17, 2022
Python SCript to scrape members from a selected Telegram group.

A python script to scrape all the members in a telegram group anad save in a CSV file. REGESTRING Go to this link https://core.telegram.org/api/obtain

Gurjeet Singh 7 Dec 1, 2022
Telegram bot to scrape images from the reddit universe

Telegram bot to scrape images from the reddit universe

XD22 3 Sep 30, 2022