Using the test code below, it seems that the filter options isn't working properly. If i run the code, all type of bikes show in the results.

import pycraigslist

bike_filters = {
    'posted_today': True,
    'bundle_duplicates': True,
    'make': 'Harley',
    'model': 'Softail'
}

bikes = pycraigslist.forsale.mca(site='miami', area='brw', filters=bike_filters)

for bike in bikes.search(limit=5):
    print(bike)

From the beginning, I was using the following to get the filters.

import pycraigslist

print(pycraigslist.forsale.mca.get_filters())

I needed to verify my filters again and when I issued the command, it replied:

Traceback (most recent call last):
  File "C:\Users\mgpd\PycharmProjects\MOlivo\py_clist.py", line 3, in <module>
    print(pycraigslist.forsale.mca.get_filters())
TypeError: get_filters() missing 1 required positional argument: 'self'

Hi,

Your API does not allow me me to use some categories (subcategories?) . For example I specifically want to use the "mcy" category which a subset of the "mca" category. The API return an error if I use this:

bikes = pycraigslist.forsale.mcy(site='miami', area='brw', filters=bike_filters)

This is the error return.

Traceback (most recent call last):
File "C:/Users/mgpd/PycharmProjects/molivo/py_clist.py", line 11, in <module>
bikes = pycraigslist.forsale.mcy(site='miami', area='brw', filters=bike_filters)
AttributeError: type object 'forsale' has no attribute 'mcy'

The "mcy" category is valid. This is the motorcycles for sale by owners.

This category is returned in the data set. Here the full test script:

import pycraigslist

bike_filters = {
    'posted_today': True,
    'bundle_duplicates': True,
    'make': 'Harley',
    'model': 'Softail'
}

bikes = pycraigslist.forsale.mca(site='miami', area='brw', filters=bike_filters)

for bike in bikes.search(limit=5):
    print(bike)

Using the script below yields different results based on the environment. When used with Google Collab (python 3.8.8) the limit function works fine. When used in my PyCharm (also python 3.8.8) it returns exactly 3 times the limit requested. In this case 15 results are returned in PyCharm.

import pycraigslist

bike_filters = {
    'posted_today': True,
    'bundle_duplicates': True,
    'make': 'Harley',
    'model': 'Softail'
}

bikes = pycraigslist.forsale.mca(site='miami', area='brw', filters=bike_filters)

for bike in bikes.search(limit=5):
    print(bike)

Let it be known, that my PyCharm is on Windows 10.

Is there an easy way to verify if a site (e.g. sfbay, raleigh, etc.) is valid prior to searching via pycraigslist?

Consider the example below:

import pycraigslist

def search_miatas(site: str) -> bool:
    miatas = pycraigslist.forsale.cta(site=site, query="mx5|miata")
    for miata in miatas.search(limit=1):
        print(miata)

If I call this function using a known good site, then pycraigslist works as expected.

search_miatas("raleigh")

>>> {'country': 'US', 'region': 'NC', 'site': 'raleigh', 'area': '', 'category': 'ctd', 'id': '7358600557', 'repost_of': '7342453821', 'last_updated': '2021-07-30 15:51', 'title': '2016 Mazda MX-5 Miata Grand Touring 2dr Convertible 6M 8000 Miles', 'neighborhood': 'Durham', 'price': '$27,495', 'url': 'https://raleigh.craigslist.org/ctd/d/durham-2016-mazda-mx-miata-grand/7358600557.html'}

However, if I call this function with a known incorrect site, then pycraigslist eventually raises MaximumRequestsError.

search_miatas("test")

>>> MaximumRequestsError: Maximum requests attempted - check network connection.

This makes logical sense, because in the background, pycraigslist is trying to hit https://test.craigslist.org, which doesn't exist, so httpx can't connect, so tenacity raises TryError, which pycraigslist re-raises as MaximumRequestsError.

The problem is that the root cause of MaximumRequestsError could be unclear if I'm accepting site input from an unsanitized source (meaning, raw user input). From what I can tell, there's not an easy way to verify whether MaximumRequestsError is raised because there's a legitimate network connectivity issue, or if the user-defined site is invalid.

I can think of two ways to solve this, assuming it's not solved already:

1. Whenever a pycraigslist.api.BaseAPI object is instantiated, prior to fetching additional filters, parse the list of Craigslist sites, validate that the site parameter is in the list of sites, and raise an exception if it's not. Downside of this approach is increased execution time (meaning, decreased performance) whenever instantiating a new object.
2. Keep an internal mapping of valid site subdomains and perform the same validation described above. Downside of this approach is, whenever Craigslist adds a new site, you have to throw together a new release to add support for it.

Ira,

Is there a way we could get a result count before running the loop? Does not need an exact number. It just could be a False/True situation. I don't know where you would put this, but if the has_results = True you run the loop.

Not a deal-breaker, but it would be nice.

Thanks

Fix #8 by adding a new custom exception, InvalidFilterValue, which will allow for more graceful error handling in scenarios where a Craigslist search filter does not have a valid value.

Also added a few basic unit tests for some underlying filter argument/value parsing functions.

Right now, the pycraigslist.query.filters.parse_filters() function throws ValueError with a string indicating a filter has an incorrect value.

It would be nice if we can create a custom exception inherited from ValueError that takes in the filter name and incorrect value as parameters. This promotes more graceful error-handling in applications using pycraigslist.

I'm thinking something like this in pycraigslist.exceptions:

class InvalidFilterValue(ValueError):
    def __init__(self, name, value):
        self.name = name
        self.value = value

Dependabot Preview will be shut down on August 3rd, 2021. In order to keep getting Dependabot updates, please merge this PR and migrate to GitHub-native Dependabot before then.

Dependabot has been fully integrated into GitHub, so you no longer have to install and manage a separate app. This pull request migrates your configuration from Dependabot.com to a config file, using the new syntax. When merged, we'll swap out dependabot-preview (me) for a new dependabot app, and you'll be all set!

With this change, you'll now use the Dependabot page in GitHub, rather than the Dependabot dashboard, to monitor your version updates, and you'll configure Dependabot through the new config file rather than a UI.

If you've got any questions or feedback for us, please let us know by creating an issue in the dependabot/dependabot-core repository.

Learn more about migrating to GitHub-native Dependabot

Please note that regular @dependabot commands do not work on this pull request.

The following Python code was returning accurate results for several months. In tests with results showing up properly on craigslist, the library is returning 0 results using the same query.

city = "newyork"
result = pycraigslist.gigs.cpg(site=city, posted_today=True, search_distance=100)

It appears Craigslist has added a redirect to the search resulting in adding something like this to the URL:

#search=1~list~0~0

Might this be affecting the library? I also noticed that when modifying searchNearby to 0 in the URL, the page returns results, but this is not able to be set using this library.