Tools for the extraction of OpenStreetMap street network data

Description of the bug

Geopandas 0.7 changed the CRS type from a str to a pyproj.CRS class instance in this commit. This causes the check in https://github.com/UDST/osmnet/blob/ec2dc954673cfdbef48e96ea805980a8a0a95fe7/osmnet/load.py#L488 to fail with the below error.

Environment

• Python version: 3.8

• OSMnet version: 0.1.5

Paste the code that reproduces the issue here:

osm.pdna_network_from_bbox(lat_min=ymin, lng_min=xmin, lat_max=ymax, lng_max=xmax)

Paste the error message (if applicable):

  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/netbuffer/core/network.py", line 90, in get_osm_network
    network = osm.pdna_network_from_bbox(lat_min=ymin,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/pandana/loaders/osm.py", line 49, in pdna_network_from_bbox
    nodes, edges = network_from_bbox(lat_min=lat_min, lng_min=lng_min,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 850, in network_from_bbox
    nodes, ways, waynodes = ways_in_bbox(
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 654, in ways_in_bbox
    osm_net_download(lat_max=lat_max, lat_min=lat_min, lng_min=lng_min,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 136, in osm_net_download
    geometry_proj, crs_proj = project_geometry(polygon,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 448, in project_geometry
    gdf_proj = project_gdf(gdf, to_latlong=to_latlong)
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 485, in project_gdf
    if (gdf.crs is not None) and ('proj' in gdf.crs) \
TypeError: argument of type 'CRS' is not iterable

Description of the bug

I have done the same over different regions of India and USA, I didn't get this issue untill trying over Banglore... Any thoughts and answers are most welcomed, Thanks!

OSM data (optional)

bbox = [12.8881, 77.5079, 12.9918, 77.6562]

Environment

• Operating system: Windows 10 pro

• Python version: 3.7.6

• OSMnet version: 0.1.5

• OSMnet required packages versions (optional): pandana 0.4.4, pandas 0.25.3, geopandas 0.6.3

Paste the code that reproduces the issue here:

import pandas as pd, geopandas as gpd
df = pd.DataFrame(
    {'id':[1522786113, 2227865111, 309601717, 1513928857, 2220792136, 6455354942],
        'Name': ['Neil', 'Nitin', 'Mukesh','Alpha','Beta','Office'],
     'Area': ['Valsad', 'Silvasa', 'Daman','Rajkot','Dui','Vapi'],
     'lat': [12.956550, 12.950360, 12.912047,12.955546,12.939653,12.928109],
     'lon': [77.540640, 77.581135, 77.586969,77.529658,77.542523,77.639337]})
pois = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.lon, df.lat))

network = osm.network_from_bbox(bbox[0], bbox[1], bbox[2], bbox[3],network_type='drive')
network=pandana.Network(network[0]["x"],network[0]["y"], network[1]["from"], network[1]["to"],network[1][["distance"]])
lcn = network.low_connectivity_nodes(impedance=1000, count=10, imp_name='distance')

network.precompute(distance + 1)
network.init_pois(num_categories=1, max_dist=distance, max_pois=7)
network.set_pois(category='my_amenity', x_col=pois['lon'], y_col=pois['lat'])
access = network.nearest_pois(distance=distance, category='my_amenity', num_pois=7)

bbox_aspect_ratio = (bbox[2] - bbox[0]) / (bbox[3] - bbox[1])
fig_kwargs = {'facecolor':'w', 
              'figsize':(15, 15 * bbox_aspect_ratio)}

# keyword arguments to pass for scatter plots
plot_kwargs = {'s':5, 
               'alpha':0.9, 
               'cmap':'viridis_r', 
               'edgecolor':'none'}

n = 1
bmap, fig, ax = network.plot(access[n], bbox=bbox, plot_kwargs=plot_kwargs, fig_kwargs=fig_kwargs)
ax.set_facecolor('k')
ax.set_title('Walking distance (m) to nearest boi around Bangalore-India', fontsize=15)
fig.savefig('images/accessibility.png', dpi=200, bbox_inches='tight')
plt.show()

Paste the error message (if applicable):

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-d557c4f5993e> in <module>
      1 # plot the distance to the nth nearest amenity
      2 n = 1
----> 3 bmap, fig, ax = network.plot(access[n], bbox=bbox, plot_kwargs=plot_kwargs, fig_kwargs=fig_kwargs)
      4 ax.set_facecolor('k')
      5 ax.set_title('Walking distance (m) to nearest {} around Berkeley/Oakland'.format(amenity), fontsize=15)

~\miniconda3\envs\ox\lib\site-packages\pandana\network.py in plot(self, data, bbox, plot_type, fig_kwargs, bmap_kwargs, plot_kwargs, cbar_kwargs)
    457         bmap = Basemap(
    458             bbox[1], bbox[0], bbox[3], bbox[2], ax=ax, **bmap_kwargs)
--> 459         bmap.drawcoastlines()
    460         bmap.drawmapboundary()
    461 

~\miniconda3\envs\ox\lib\site-packages\mpl_toolkits\basemap\__init__.py in drawcoastlines(self, linewidth, linestyle, color, antialiased, ax, zorder)
   1849         # get current axes instance (if none specified).
   1850         ax = ax or self._check_ax()
-> 1851         coastlines = LineCollection(self.coastsegs,antialiaseds=(antialiased,))
   1852         coastlines.set_color(color)
   1853         coastlines.set_linestyle(linestyle)

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in __init__(self, segments, linewidths, colors, antialiaseds, linestyles, offsets, transOffset, norm, cmap, pickradius, zorder, facecolors, **kwargs)
   1357             **kwargs)
   1358 
-> 1359         self.set_segments(segments)
   1360 
   1361     def set_segments(self, segments):

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in set_segments(self, segments)
   1372             _segments = self._add_offsets(_segments)
   1373 
-> 1374         self._paths = [mpath.Path(_seg) for _seg in _segments]
   1375         self.stale = True
   1376 

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in <listcomp>(.0)
   1372             _segments = self._add_offsets(_segments)
   1373 
-> 1374         self._paths = [mpath.Path(_seg) for _seg in _segments]
   1375         self.stale = True
   1376 

~\miniconda3\envs\ox\lib\site-packages\matplotlib\path.py in __init__(self, vertices, codes, _interpolation_steps, closed, readonly)
    128         if vertices.ndim != 2 or vertices.shape[1] != 2:
    129             raise ValueError(
--> 130                 "'vertices' must be a 2D list or array with shape Nx2")
    131 
    132         if codes is not None:

ValueError: 'vertices' must be a 2D list or array with shape Nx2

Suggestion

User should be able to supply custom OSM filter when calling network_from_bbox

The two predefined filter choices for 'walk' and 'drive' are helpful, but somewhat limiting. Adding this would allow for much greater generalizability.

• addresses: #26
• update requirements to: geopandas >= 0.8.2 and shapely >= 1.6
• drops Python 3.7 from tests for now due to Travis build error
• updates contribution guidelines

This PR finalizes the v0.1.6 release by merging dev into master. See PRs #22 and #21 for more info.

Distribution checklist:

• [x] released on PyPI: https://pypi.org/project/osmnet/
• [x] released on Conda Forge: https://anaconda.org/conda-forge/osmnet
• [x] live docs updated: https://udst.github.io/osmnet

Installation tests (because we've dropped Python 2.7 support in this version):

• [x] Pip py38 -> OSMnet 0.1.6
• [x] Pip py27 -> OSMnet 0.1.5
• [x] Conda py38 -> OSMnet 0.1.6
• [x] Conda py27 -> OSMnet 0.1.5

This PR prepares the OSMnet v0.1.6 release, which provides compatibility with GeoPandas v0.7 and later.

The easiest way to implement that was for this version of OSMnet to require GeoPandas v0.7+, which means that it no longer supports Python 2.7 or Win32. (OSMnet v0.1.5 continues to install and run fine on older platforms, for anyone who needs it.)

cc @sablanchard @knaaptime @blakerosenthal

Previously merged:

• PR #21

Changes in this PR:

CI cleanup

• Pins pytest-cov to an earlier version to resolve an installation incompatibility with Coveralls, similar to what's described here
• Updates pycodestyle settings (the defaults must have changed)
• In Travis, adds Python 3.8 and drops 2.7
• Updates the AppVeyor script to be cleaner and more targeted
• Switches AppVeyor from Win32 to Win64

Other changes

• Bumps the Pandas requirement to correspond to GeoPandas 0.7
• Adds a setting in setup.py to require Python 3 (following instructions here)
• Updates copyright year in license and docs
• Updates version numbers in setup, init, and docs
• Adds version info to the main doc page
• Cleans up and documents the doc building procedure
• Updates contribution guide with additional info drawn from other UDST repos
• Renames HISTORY to CHANGELOG and adds an entry for this release
• Cleans up some cruft in the repo, like references to ez_setup

Testing:

• [x] Unit tests are passing on Mac (local), Linux (travis), and Windows (appveyor)

Next steps:

• Merge this to dev
• Release on PyPI
• Update conda-forge feedstock
• Update live docs
• Merge to master
• Tag release on GitHub
• Confirm that Pip and Conda automatically install previous version in Python 2.7

And we should probably go through the other deprecation warnings when we have a chance (@sablanchard), and fix things before the library breaks again!

This PR makes some small changes to the packaging.

I added a MANIFEST.in file that lists the changelog and the license. This will include those files in the source distribution when we put together releases for pypi (and by extension conda-forge, which gets the files from pypi). The license file in particular is required by conda-forge.

I also added a .txt file extension to the license file because it seems tidier.

No substantive changes to the code.

EDITED TO ADD:

• fixed the version number in __init__.py, which hadn't been updated for the last release (this won't show up in pypi and conda-forge though)
• streamlined the travis tests to match the standard install process (now only installs requirements that are specified in setup.py or requirements-dev.txt)
• added python 3.6 and 3.7 to test suite and affirmed support in setup.py
• fixed tests that were failing due to changes in OSM database

Unit tests in Python 3.6.11 for osmnet v1.6.0 are returning misc DeprecationWarning and FutureWarnings when using geopandas 0.9, shapely 1.7.1, and pandas 1.1.0:

1. DeprecationWarning: Flags not at the start of the expression '//(?s)(.*?)/' from https://github.com/UDST/osmnet/blob/dev/osmnet/load.py#L237, suggest to change (r'//(?s)(.*?)/', url)[0] to (r'(?s)//(.*?)/', url)[0]

2. FutureWarning: Assigning CRS to a GeoDataFrame without a geometry column is now deprecated and will not be supported in the future. from: https://github.com/UDST/osmnet/blob/dev/osmnet/load.py#L445 suggest refactoring function to accept new schema.

3. FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6 return _prepare_from_string(" ".join(pjargs)) when using 'init' to initialize a projection, suggest using latest crs specification such as "EPSG:4326"

4. FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead. import pandas.util.testing as pdt from: https://github.com/UDST/osmnet/blob/dev/osmnet/tests/test_load.py#L2 suggest removing use of pdt and instead use .equals()

The last OSMnet release ended support for Python 2.7 (see PR #23). I didn't get the distribution details quite right, and in some cases Pip in py27 was still trying to install the new version. (This came up in the UrbanAccess Travis tests.)

This PR documents what was going on and corrects the release instructions. The key is that -- in addition to specifying python_requires in setup.py -- you need to build and upload a non-universal wheel file for PyPI.

Details

Here's some more info: https://packaging.python.org/guides/distributing-packages-using-setuptools/#wheels

Universal wheels imply support for both Python 2 and Python 3. You build them with

python setup.py sdist bdist_wheel --universal

And the filenames look like osmnet-0.1.6-py2.py3-none-any.whl.

To make it clear that the package no longer supports Python 2, you need a non-universal pure Python wheel. You build it with

python setup.py sdist bdist_wheel

And the filename looks like osmnet-0.1.6-py3-none-any.whl. (If your active environment is Python 3, it creates a wheel for Python 3.)

Note that the wheels are the bdist_wheel part of the command -- the sdist part creates a tarball like osmnet-0.1.6.tar.gz, and we upload both to PyPI. Conda Forge uses the tarball for the conda release, and probably in some cases users install from it too.

Resolution

Earlier today I uploaded a non-universal wheel for OSMnet v0.1.6 to PyPI, and deleted the universal one. So if you've been having problems it should be fixed now.

It looks like osmnet v0.1.5 was never released to the conda-forge channel and the https://github.com/UDST/osmnet/blob/master/osmnet/init.py#L3 was not updated to match the latest version number v0.1.5.

Problem

People sometimes get errors using setuptools to install packages that list OSMNet as a requirement.

For example, OSMNet is a requirement in Pandana's setup.py file. If you install Pandana on OS X or Linux using python setup.py develop, and OSMNet is not yet installed, you get the error below.

Solution

I poked around, and it seems like the problem is that the initial 0.1.0 release of OSMNet on pypi.org included files that no longer conform to the standard naming spec. This causes an error when setuptools tries to find an appropriate version to install.

There are several newer releases, none of which break the API in any way, so I went ahead and removed 0.1.0 from pypi. This immediately fixed the problem on my machine.

Let me know if you see any more issues related to this.

Original error message

Searching for osmnet>=0.1.2
Reading https://pypi.org/simple/osmnet/
Traceback (most recent call last):
  File "setup.py", line 133, in <module>
    'License :: OSI Approved :: GNU Affero General Public License v3'
  File "/anaconda3/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/anaconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/develop.py", line 38, in run
    self.install_for_development()
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/develop.py", line 156, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 752, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
    not self.always_copy, self.local_index
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 658, in fetch_distribution
    self.find_packages(requirement)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 491, in find_packages
    self.scan_url(self.index_url + requirement.unsafe_name + '/')
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 831, in scan_url
    self.process_url(url, True)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 373, in process_url
    self.process_url(link)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 335, in process_url
    dists = list(distros_for_url(url))
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 99, in distros_for_url
    for dist in distros_for_location(url, base, metadata):
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 118, in distros_for_location
    wheel = Wheel(basename)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/wheel.py", line 64, in __init__
    raise ValueError('invalid wheel name: %r' % filename)
ValueError: invalid wheel name: 'osmnet-0.1.0-py2.py3-any.whl'

• updates to support package builds of Python 3.9 and 3.10
• updates to address Shapely depreciation of iterating over multi-part geometries
• bump up minimum supported version of geopandas and shapely
• minor update to unit tests to update expected results using latest OSM data pulls
• minor readme and doc updates

It would be nice to have some of the bug fixes since the latest version.

Explanation: I was using pandana package which relies on osmnet, and am getting an error that is fixed by this commit, which was made after 0.6.1 https://github.com/UDST/osmnet/commit/6706683390ff7019bad0ba1eee82384fbae6b38d

Thanks for your time.