Hi there,

first of all, great library, it's been really useful to me!

Now my issue, after experiementing with the public endpoints I've setup a personal overpass endpoint to escape the wait times and not over utilize public goods. I've followed along the instructions on: https://wiki.openstreetmap.org/wiki/Overpass_API/Installation Installing Release 0.7.58

Pointing the library to my endpoint via: Overpass(endpoint='<MyEndpoint>')

I ended up with the following error (happy to share a more detailed trace if needed): Exception: [overpass] could not fetch or interpret status of the endpoint

After some analysis it seems I'm failing in the libraries _waitForReady function because my status response has more rows and therefore the statusString[] indexes are off.

Status Response Connected as: 3232235754 Current time: 2022-06-01T15:19:28Z Announced endpoint: none Rate limit: 0 Currently running queries (pid, space limit, time limit, start time):

Can the function be changed to work line independent and detect required infos smarter? I'm just starting out so I don't want to mess with the libraries code.

Cheers!

Hi there,

I am new to OSM and wanted to play around with it in a Jupiter Notebook.

I checked out the readme and wanted to implement the example. But somehow it does not do what it is supposed to. Please check out the error messages in the attachment.

Any suggestion is greatly appreciated.

Thanks Pascal

If a query results in an error (e.g., a timeout) even re-running the query with changed parameters (e.g., a different timeout value) immediately bails out. I have not checked if timeouts are the only problem but in general the cache should be invalidated in those cases.

(I am building the query strings manually; using 0.3.0 via pip)

This would make querying within features with known ids/without names simpler by including logic to calculate the feature's corresponding area id.

Example usage:

• How many footways are in a given park polygon? from OSMPythonTools.overpass import overpassQueryBuilder, Overpass overpass = Overpass() query = overpassQueryBuilder(area='w290617734', elementType='way', selector='highway=footway', out='body') result = overpass.query(query) result.countElements() #16

Hi, first of all thank you very much for all your work on this package.

While insalling this package, I realised that it introduces a dependency on pytest and pytest-sugar (see this line and this line).

Are those dependencies needed for end users, or should they be distributed somewhere else for developer of this library only? I'm asking this question because installing the package produces a side-effect for me which is modifying my pytest interface.

Thank you for your time.

The library often handles exception itself, e.g., cacheObject's __query(), by excepting, logging, and then raising a newly created generic Exception object. For example very specific exceptions raised by urllib get converted into this generic type. It thus becomes impossible to differentiate between different errors on the user side, e.g., it one cannot even react differently if the user hits ctrl+c while a transfer is in progress. As is it is completely impossible to do meaningful error handling without inspection or other tricks AFAICT.

It would be much better to leave the respective error handling to the library user instead. You can still add useful information to the exception (and log stuff if you like), cf. https://wiki.python.org/moin/HandlingExceptions

With the following:

from OSMPythonTools.overpass import Overpass

op = Overpass() q = 'relation(110796); out body;' r = op.query(q) e = r.elements()[0] e.geometry()

I get the following:

Exception: [OSMPythonTools.Element] Cannot build geometry: cannot find outer ring for inner ring. (relation/110796)

Is there a way to download all nodes of requested ways during the overpass query?

I have this query: query = f'''area({area_id})->.searchArea; ( rel(area.searchArea); way(area.searchArea); node(area.searchArea); )->.result; .result out meta; .result >->.recurseresult; .recurseresult out meta;'''

result = overpass.query(query) In my understanding, even if all nodes of a way w are in results, the first time I call w.nodes() it will download all of the nodes.

I would like:

1. query() to download all data so that no other http request are made.
2. nodeRefs() or similar to receive node ids only (currently this info is buried into _json)

Is there such functionality or is planned?

Thanks, Angelo

This may not be a feature desired by the authors, but I thought it might be helpful to pass other parameters when querying the nominatim API (Nominatim search API docs).

Doing:

query = overpassQueryBuilder(area=3600007407, elementType='way',    selector='"railway"="rail"', out='body')
a=overpass.query(query)

len(a.ways())
Out[30]: 751

len(a.elements())
Out[28]: 751

but doing on http://overpass-api.de/query_form.html way(area:3600007407)[railway=rail]; out count;

<count id="0">
    <tag k="nodes" v="0"/>
    <tag k="ways" v="764"/>
    <tag k="relations" v="0"/>
    <tag k="areas" v="0"/>
    <tag k="total" v="764"/>
  </count>

overpy also gives 764 elements.

It seems that missing ways were recently modified like 852538015: https://www.openstreetmap.org/way/852538015 edited 3 month ago

Do you have any idea ?

Hello!

I am trying to query for objects (nodes and ways) that are around some location represented as a (lat, lon) pair. My query is

query_obj = '(node(around:10000,{0},{1});way(around:10000,{0},{1});); out body;'.format(lat, lon)
overpass.query(query_obj, timeout=5)

And while it works fine with some (lat, lon) pairs, e.g. (52.6851, -0.470191), it throws an exception with others, e.g. (54.59298,-1.48307).

This is what I am getting

What could be the reason? And do you have any idea how to avoid that?

Thanks in advance!

I recently updated from version 0.2.8 to 0.3.5 and noticed a significant increase in querying time for overpass queries with the new version. Testing around with different longer queries showed an increase in time by a factor of 100-150x. Dug a bit in the source code and it seems to be an issue with the ApiResult class (in api.py) - and how it was rewritten since version 0.3.3. More specifically, how BeautifulSoup and the souphistory is handled now is decreasing OSMPythonTools performance significantly.

For reference:

new, slow version (>=0.3.3)

class ApiResult(Element):
    def __init__(self, xml, queryString, params, cacheMetadata=None, shallow=False, history=False):
        self._isValid = (xml != {} and xml is not None)
        self._xml = xml
        self._soup2 = None
        soup = None
        soupHistory = None
        if self._isValid:
            self._soup2 = BeautifulSoup(xml, 'xml').find('osm')
            soupHistory = self._soup2.find_all(['node', 'way', 'relation'])
            if len(soupHistory) > 0:
                soup = soupHistory[-1]
        super().__init__(cacheMetadata, soup=soup, soupHistory=soupHistory if history else None, shallow=shallow)
        self._queryString = queryString
        self._params = params

older, more efficient version (<0.3.3)

class ApiResult(Element):
    def __init__(self, xml, queryString, shallow=False):
        self._isValid = (xml != {} and xml is not None)
        self._xml = xml
        self._soup = None
        soupElement = None
        if self._isValid:
            self._soup = BeautifulSoup(xml, 'xml')
            if len(self._soup.find_all('node')) > 0:
                soupElement = self._soup.node
            if len(self._soup.find_all('way')) > 0:
                soupElement = self._soup.way
            if len(self._soup.find_all('relation')) > 0:
                soupElement = self._soup.relation
        super().__init__(soup=soupElement, shallow=shallow)
        self._queryString = queryString

For example:

from OSMPythonTools.api import Api
api = Api()
way = api.query('relation/3629242')
way.centerLat()
way.centerLon()

I understand that it is probably not trivial and some computation would have to be done in the background, but either remove those methods when they cannot be filled with some meaningful data or compute the result?

When i import the Api by this command:

from OSMPythonTools.api import Api

I have this exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/evblinov/Desktop/Projects/candidates_gateway/venv/lib/python3.10/site-packages/OSMPythonTools/api.py", line 4, in <module>
    from OSMPythonTools.internal.cacheObject import CacheObject
  File "/Users/evblinov/Desktop/Projects/candidates_gateway/venv/lib/python3.10/site-packages/OSMPythonTools/internal/cacheObject.py", line 5, in <module>
    import ujson
ImportError: dlopen(/Users/evblinov/Desktop/Projects/candidates_gateway/venv/lib/python3.10/site-packages/ujson.cpython-310-darwin.so, 0x0002): tried: '/Users/evblinov/Desktop/Projects/candidates_gateway/venv/lib/python3.10/site-packages/ujson.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))

Having shallow parameter on True for overpass results in passing just a boolean as dict to _rawToResult function and ends up with

File .../.venv/lib/python3.8/site-packages/OSMPythonTools/overpass.py:144, in OverpassResult.__get(self, prop)
    143 def __get(self, prop):
--> 144     return self._json[prop] if prop in self._json else None

https://github.com/mocnik-science/osm-python-tools/blob/695ac4f6b9ee1c6ea4071b6dce28160687fd18c8/OSMPythonTools/internal/cacheObject.py#L50

https://github.com/mocnik-science/osm-python-tools/blob/695ac4f6b9ee1c6ea4071b6dce28160687fd18c8/OSMPythonTools/overpass.py#L77-L78

https://github.com/mocnik-science/osm-python-tools/blob/695ac4f6b9ee1c6ea4071b6dce28160687fd18c8/OSMPythonTools/overpass.py#L124-L126

https://github.com/mocnik-science/osm-python-tools/blob/695ac4f6b9ee1c6ea4071b6dce28160687fd18c8/OSMPythonTools/overpass.py#L143-L144

Maybe I didn't understand how to do it, but it would be very useful to be able to pass a language-related parameter in Nominatim. Something similar to: loc = nominatim.query(*coordinates, reverse=True, wkt=True, language='en' )

I was trying to find all runways within a bounding box, and noticed that I get every runway (mapped as a way with aeroway=runway) twice. Here's a test code:

from OSMPythonTools import overpass
query = overpass.overpassQueryBuilder(bbox=[-6.05, 144.95, -6, 145], elementType="way",
                                                                 selector='"aeroway"="runway"', out="body", includeCenter=True)
result = overpass.Overpass().query(query).ways()
for e in result:
    print(e, e.tags())

which prints:

[overpass] downloading data: [timeout:25][out:json];(way["aeroway"="runway"](-6.05,144.95,-6,145);); out center; out body;
<OSMPythonTools.element.Element object at 0x7fb7a11704c0> {'aeroway': 'runway', 'ele': '1516', 'length': '1015', 'name': '03/21', 'source': 'Bing', 'surface': 'paved'}
<OSMPythonTools.element.Element object at 0x7fb7a11700d0> {'aeroway': 'runway', 'ele': '1516', 'length': '1015', 'name': '03/21', 'source': 'Bing', 'surface': 'paved'}

You can see that I got two different Elements holding the exact same data. Here's the test area in OSM And Overpass-turbo shows that there is in fact only one way tagged ´aeroway=runway´ in that area.