Fiona reads and writes geographic data files

Overview

Fiona

Fiona reads and writes geographic data files and thereby helps Python programmers integrate geographic information systems with other computer systems. Fiona contains extension modules that link the Geospatial Data Abstraction Library (GDAL).

https://travis-ci.org/Toblerity/Fiona.png?branch=master https://ci.appveyor.com/api/projects/status/github/Toblerity/Fiona?svg=true https://coveralls.io/repos/Toblerity/Fiona/badge.png

Fiona is designed to be simple and dependable. It focuses on reading and writing data in standard Python IO style and relies upon familiar Python types and protocols such as files, dictionaries, mappings, and iterators instead of classes specific to GDAL's OpenGIS Reference Implementation (OGR). Fiona can read and write real-world data using multi-layered GIS formats and zipped virtual file systems and integrates readily with other Python GIS packages such as pyproj, Rtree, and Shapely.

Fiona is supported only on CPython versions 3.6+.

Why the name "Fiona"? Because Fiona is OGR's neat and nimble API for Python programmers. And a Shrek reference made us laugh.

For more details, see:

Usage

Collections

Records are read from and written to file-like Collection objects returned from the fiona.open() function. Records are mappings modeled on the GeoJSON format. They don't have any spatial methods of their own, so if you want to do anything fancy with them you will probably need Shapely or something like it. Here is an example of using Fiona to read some records from one data file, change their geometry attributes, and write them to a new data file.

import fiona

# Open a file for reading. We'll call this the "source."

with fiona.open('tests/data/coutwildrnp.shp') as src:

    # The file we'll write to, the "destination", must be initialized
    # with a coordinate system, a format driver name, and
    # a record schema.  We can get initial values from the open
    # collection's ``meta`` property and then modify them as
    # desired.

    meta = src.meta
    meta['schema']['geometry'] = 'Point'

    # Open an output file, using the same format driver and
    # coordinate reference system as the source. The ``meta``
    # mapping fills in the keyword parameters of fiona.open().

    with fiona.open('test_write.shp', 'w', **meta) as dst:

        # Process only the records intersecting a box.
        for f in src.filter(bbox=(-107.0, 37.0, -105.0, 39.0)):

            # Get a point on the boundary of the record's
            # geometry.

            f['geometry'] = {
                'type': 'Point',
                'coordinates': f['geometry']['coordinates'][0][0]}

            # Write the record out.

            dst.write(f)

# The destination's contents are flushed to disk and the file is
# closed when its ``with`` block ends. This effectively
# executes ``dst.flush(); dst.close()``.

Reading Multilayer data

Collections can also be made from single layers within multilayer files or directories of data. The target layer is specified by name or by its integer index within the file or directory. The fiona.listlayers() function provides an index ordered list of layer names.

for layername in fiona.listlayers('tests/data'):
    with fiona.open('tests/data', layer=layername) as src:
        print(layername, len(src))

# Output:
# ('coutwildrnp', 67)

Layer can also be specified by index. In this case, layer=0 and layer='test_uk' specify the same layer in the data file or directory.

for i, layername in enumerate(fiona.listlayers('tests/data')):
    with fiona.open('tests/data', layer=i) as src:
        print(i, layername, len(src))

# Output:
# (0, 'coutwildrnp', 67)

Writing Multilayer data

Multilayer data can be written as well. Layers must be specified by name when writing.

with open('tests/data/cowildrnp.shp') as src:
    meta = src.meta
    f = next(src)

with fiona.open('/tmp/foo', 'w', layer='bar', **meta) as dst:
    dst.write(f)

print(fiona.listlayers('/tmp/foo'))

with fiona.open('/tmp/foo', layer='bar') as src:
    print(len(src))
    f = next(src)
    print(f['geometry']['type'])
    print(f['properties'])

    # Output:
    # ['bar']
    # 1
    # Polygon
    # OrderedDict([('PERIMETER', 1.22107), ('FEATURE2', None), ('NAME', 'Mount Naomi Wilderness'), ('FEATURE1', 'Wilderness'), ('URL', 'http://www.wilderness.net/index.cfm?fuse=NWPS&sec=wildView&wname=Mount%20Naomi'), ('AGBUR', 'FS'), ('AREA', 0.0179264), ('STATE_FIPS', '49'), ('WILDRNP020', 332), ('STATE', 'UT')])

A view of the /tmp/foo directory will confirm the creation of the new files.

$ ls /tmp/foo
bar.cpg bar.dbf bar.prj bar.shp bar.shx

Collections from archives and virtual file systems

Zip and Tar archives can be treated as virtual filesystems and Collections can be made from paths and layers within them. In other words, Fiona lets you read and write zipped Shapefiles.

for i, layername in enumerate(
        fiona.listlayers('zip://tests/data/coutwildrnp.zip'):
    with fiona.open('zip://tests/data/coutwildrnp.zip', layer=i) as src:
        print(i, layername, len(src))

# Output:
# (0, 'coutwildrnp', 67)

Fiona can also read from more exotic file systems. For instance, a zipped shape file in S3 can be accessed like so:

with fiona.open('zip+s3://mapbox/rasterio/coutwildrnp.zip') as src:
    print(len(src))

# Output:
# 67

Fiona CLI

Fiona's command line interface, named "fio", is documented at docs/cli.rst. Its fio info pretty prints information about a data file.

$ fio info --indent 2 tests/data/coutwildrnp.shp
{
  "count": 67,
  "crs": "EPSG:4326",
  "driver": "ESRI Shapefile",
  "bounds": [
    -113.56424713134766,
    37.0689811706543,
    -104.97087097167969,
    41.99627685546875
  ],
  "schema": {
    "geometry": "Polygon",
    "properties": {
      "PERIMETER": "float:24.15",
      "FEATURE2": "str:80",
      "NAME": "str:80",
      "FEATURE1": "str:80",
      "URL": "str:101",
      "AGBUR": "str:80",
      "AREA": "float:24.15",
      "STATE_FIPS": "str:80",
      "WILDRNP020": "int:10",
      "STATE": "str:80"
    }
  }
}

Installation

Fiona requires Python versions 3.6+ and GDAL version 1.11-3.0. To build from a source distribution you will need a C compiler and GDAL and Python development headers and libraries (libgdal1-dev for Debian/Ubuntu, gdal-dev for CentOS/Fedora).

To build from a repository copy, you will also need Cython to build C sources from the project's .pyx files. See the project's requirements-dev.txt file for guidance.

The Kyngchaos GDAL frameworks will satisfy the GDAL/OGR dependency for OS X, as will Homebrew's GDAL Formula (brew install gdal).

Python Requirements

Fiona depends on the modules six, cligj, and munch. Pip will fetch these requirements for you, but users installing Fiona from a Windows installer must get them separately.

Unix-like systems

Assuming you're using a virtualenv (if not, skip to the 4th command) and GDAL/OGR libraries, headers, and gdal-config program are installed to well known locations on your system via your system's package manager (brew install gdal using Homebrew on OS X), installation is this simple.

$ mkdir fiona_env
$ virtualenv fiona_env
$ source fiona_env/bin/activate
(fiona_env)$ pip install fiona

If gdal-config is not available or if GDAL/OGR headers and libs aren't installed to a well known location, you must set include dirs, library dirs, and libraries options via the setup.cfg file or setup command line as shown below (using git). You must also specify the version of the GDAL API on the command line using the --gdalversion argument (see example below) or with the GDAL_VERSION environment variable (e.g. export GDAL_VERSION=2.1).

(fiona_env)$ git clone git://github.com/Toblerity/Fiona.git
(fiona_env)$ cd Fiona
(fiona_env)$ python setup.py build_ext -I/path/to/gdal/include -L/path/to/gdal/lib -lgdal install --gdalversion 2.1

Or specify that build options and GDAL API version should be provided by a particular gdal-config program.

(fiona_env)$ GDAL_CONFIG=/path/to/gdal-config pip install fiona

Windows

Binary installers are available at http://www.lfd.uci.edu/~gohlke/pythonlibs/#fiona and coming eventually to PyPI.

You can download a binary distribution of GDAL from here. You will also need to download the compiled libraries and headers (include files).

When building from source on Windows, it is important to know that setup.py cannot rely on gdal-config, which is only present on UNIX systems, to discover the locations of header files and libraries that Fiona needs to compile its C extensions. On Windows, these paths need to be provided by the user. You will need to find the include files and the library files for gdal and use setup.py as follows. You must also specify the version of the GDAL API on the command line using the --gdalversion argument (see example below) or with the GDAL_VERSION environment variable (e.g. set GDAL_VERSION=2.1).

$ python setup.py build_ext -I<path to gdal include files> -lgdal_i -L<path to gdal library> install --gdalversion 2.1
$ set GDAL_VERSION=3.0
$ pip install --install-option=build_ext --install-option="-I<drive letter>:\\<path to gdal include files>\\include" --install-option="-lgdal_i" --install-option="-L<drive letter>:\\<path to gdal lib files>\\libs" fiona

Note: The GDAL DLL (gdal111.dll or similar) and gdal-data directory need to be in your Windows PATH otherwise Fiona will fail to work.

The Appveyor CI build uses the GISInternals GDAL binaries to build Fiona. This produces a binary wheel for successful builds, which includes GDAL and other dependencies, for users wanting to try an unstable development version. The Appveyor configuration file may be a useful example for users building from source on Windows.

Development and testing

Building from the source requires Cython. Tests require pytest. If the GDAL/OGR libraries, headers, and gdal-config program are installed to well known locations on your system (via your system's package manager), you can do this:

(fiona_env)$ git clone git://github.com/Toblerity/Fiona.git
(fiona_env)$ cd Fiona
(fiona_env)$ pip install cython
(fiona_env)$ pip install -e .[test]
(fiona_env)$ py.test

Or you can use the pep-518-install script:

(fiona_env)$ git clone git://github.com/Toblerity/Fiona.git
(fiona_env)$ cd Fiona
(fiona_env)$ ./pep-518-install

If you have a non-standard environment, you'll need to specify the include and lib dirs and GDAL library on the command line:

(fiona_env)$ python setup.py build_ext -I/path/to/gdal/include -L/path/to/gdal/lib -lgdal --gdalversion 2 develop
(fiona_env)$ py.test
Comments
  • Circular imports for Python 3.8 on Windows if DLLs are already loaded

    Circular imports for Python 3.8 on Windows if DLLs are already loaded

    WIth fiona 1.8.15, the issue is fixed now in the conda-forge builds, only the Windows py3.8 build is still failing: https://github.com/conda-forge/fiona-feedstock/pull/167

    The failure is with the general error message of partially initialized module, so doesn't give much pointers (at least to me):

    running run_test.py
    Traceback (most recent call last):
      File "D:\bld\fiona_1599200296823\test_tmp\run_test.py", line 6, in <module>
        import fiona
      File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\__init__.py", line 84, in <module>
        import fiona._loading
      File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\_loading.py", line 44, in <module>
        import fiona.ogrext
      File "fiona/ogrext.pyx", line 29, in init fiona.ogrext
      File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\env.py", line 14, in <module>
        with fiona._loading.add_gdal_dll_directories():
    AttributeError: partially initialized module 'fiona' has no attribute '_loading' (most likely due to a circular import)
    

    Originally posted by @jorisvandenbossche in https://github.com/Toblerity/Fiona/issues/941#issuecomment-686982973

    opened by rbuffat 45
  • [WIP] Add support for FlatGeobuf

    [WIP] Add support for FlatGeobuf

    It looks as Fiona is unable to write any records with the FlatGeobuf driver, but also the log does not show any errors. The metadata seems to be saved, but nothing else. Reading of FlatGeobuf seems to be fine though. I currently don't see where the problem could be.

    @tomplex @kylebarron did you had any luck with writing data?

    @bjornharrtell if I'm not mistaken, you are the author of the gdal FlatGeobuf driver. Maybe you have an idea why this is happening. But don't feel obligated to have a look at this! The relevant code for writing would be here: https://github.com/Toblerity/Fiona/blob/maint-1.8/fiona/ogrext.pyx#L1211-L1284

    opened by rbuffat 33
  • Unable to open EPSG support file gcs.csv

    Unable to open EPSG support file gcs.csv

    This issue has the same symptoms as https://github.com/mapbox/rasterio/issues/1539:

    In [1]: import fiona                                                                                                                                                                                                
    
    In [2]: ds = fiona.open('aerialway-line.shp')                                                                                                                                                                       
    
    In [3]: ds.crs                                                                                                                                                                                                      
    ERROR 4: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
    ERROR 6: No translation for an empty SRS to PROJ.4 format is known.
    Out[3]: {}
    
    In [4]: fiona.__version__                                                                                                                                                                                           
    Out[4]: '1.8.0'
    

    The open function is decorated by ensure_env_with_credentials but it seems doesn't work.

    opened by drnextgis 28
  • CRS Axis Order issues with 1.8.9

    CRS Axis Order issues with 1.8.9

    Problem Description

    When reading in: https://raw.githubusercontent.com/geopandas/geopandas/master/geopandas/tests/data/overlay/overlap/df1_df2_overlap-union.geojson

    With previous versions of Fiona, the WKT string was read in as:

    GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]
    

    However, currently (1.8.9) it is read in as:

    GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Longitude",EAST],AXIS["Latitude",NORTH]]
    

    Everything is mostly the same except for the axis order at the end.

    I believe this is due to this line: https://github.com/Toblerity/Fiona/blob/960568d2fede452a042e5c9f315fb93e3cd56baa/fiona/_crs.pyx#L65

    This is problematic for a couple of reasons:

    1. It does not preserve the original WKT string/projection and mangles it
    2. What if the user actually modified their data to be in the axis order mentioned in the WKT string? How would they tell Fiona that they did so?

    Recommended Solution

    Add an always_xy flag to crs_to_wkt() and only use it when creating a CRS to be used in the transformation. It would also be a good idea to add an always_xy flag to the transformation operations for the case where the user modified the underlying data to match the axis order specified in the CRS. For Fiona 1.8.x, the always_xy flag could be omitted from the transformation operations and just default to always_xy=True in crs_to_wkt()

    Thoughts?

    opened by snowman2 27
  • Driver not found (GPKG)

    Driver not found (GPKG)

    Expected behavior and actual behavior.

    GPKG driver is avaliable

    Steps to reproduce the problem.

    import fiona
    from fiona._drivers import GDALEnv
    env = GDALEnv()
    
    "GPKG" in env.start().drivers().keys()
    True
    

    Operating system

    Linux, Ubuntu 17.10

    Fiona version and provenance

    >>> fiona.__version__
    '1.7.10.post1'
    >>> 
    

    from pip

    opened by jachym 26
  • Add shim module and support for GDAL 3

    Add shim module and support for GDAL 3

    I propose to merge this and make a 1.8.9 release without fixing the Appveyor build so that we can ease the PROJ 4/6 pain of users. Let's tackle the Appveyor updates in the master branch.

    Summary of the PR:

    • Adds a shim module for GDAL 3, introduces two new functions (osr_get_name and osr_set_traditional_axis_mapping_strategy), and adds no-op versions of the functions to the older shims.
    • Everywhere we initialize an OGRSpatialReferenceH object using OSRImportFrom*, we set its axis order to the "traditional" one to preserve the existing axis order.
    • We disable two tests for GDAL 3 because towgs84 and wktext don't surface in GDAL's WKT output anymore.

    @hobu you're interested in this, I know. Want to give it a quick scan? I realize that some of the Cython files are archaic and a bit opaque.

    @snorfalorpagus any objections?

    opened by sgillies 19
  • Segmentation fault with simple geometry operations using Shapely and Fiona

    Segmentation fault with simple geometry operations using Shapely and Fiona

    Hi,

    I am having trouble making sense of an issue I'm having using Shapely and Fiona. I keep getting segmentation faults. even when doing very simple tasks. I've narrowed it down to being a problem having shapely and fiona loaded at the same time, which makes either package much less useful.

    I've included a minimal working example below. The first runs just fine with only shapely loaded. The second produces a segmentation fault running the same processes with shapely and fiona loaded.

    Example 1, works:

    In [1]: %paste
    from shapely import geometry
    shp = geometry.LinearRing([(0,0),(1,0),(0,1)])
    shp.centroid.coords.xy
    Out[1]: (array('d', [0.35355339059327373]), array('d', [0.35355339059327373]))
    

    Example 2, segfault:

    In [1]: %paste
    import fiona
    from shapely import geometry
    shp = geometry.LinearRing([(0,0),(1,0),(0,1)])
    shp.centroid.coords.xy
    ## -- End pasted text --
    /Users/wiar9509/anaconda/bin/python.app: line 3:   849 Segmentation fault: 11  /Users/wiar9509/anaconda/python.app/Contents/MacOS/python "$@"
    

    I am running this in iPython, using an Anaconda install. My Python information is: Python 2.7.12 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:43:17)

    My iPython version is:

    engr2-2-77-37-dhcp:~ wiar9509$ conda list ipython
    # packages in environment at /Users/wiar9509/anaconda:
    #
    ipython                   4.2.0                    py27_1  
    ipython_genutils          0.1.0                    py27_0 
    

    Fiona version:

    In [2]: fiona.__version__
    Out[2]: '1.7.0'
    

    Shapely version:

    In [3]: shapely.__version__
    Out[3]: '1.5.16'
    

    Please let me know how I might be able to solve this issue. I am excited to use these packages!

    Billy

    packaging 
    opened by armstrwa 19
  • Intermittent / transient fiona.open errors using AWS S3 + lambda

    Intermittent / transient fiona.open errors using AWS S3 + lambda

    Expected behavior and actual behavior.

    I'm using fiona==1.8.6 on AWS lambda to open shapefiles on S3. I'm seeing occasional clusters of fiona.open errors and was wondering (1) whether anyone else has seen similar behavior, and (2) whether there is anything I can do to reduce the frequency of these errors.

    The stack traces look like this:

    my_shapefile = fiona.open(shapefile_uri)
    File "/var/task/fiona/__init__.py", line 249, in open
    enabled_drivers=enabled_drivers, **kwargs)
    File "/var/task/fiona/collection.py", line 160, in __init__
    self.session.start(self, **kwargs)
    File "fiona/ogrext.pyx", line 468, in fiona.ogrext.Session.start
    File "fiona/_shim.pyx", line 73, in fiona._shim.gdal_open_vector
    fiona.errors.DriverError: '/vsis3/my_aws_bucket/my_reference_data/index.shp' not recognized as a supported file format.
    

    In case it matters, index.shp in the stack trace above is a US national index shapefile pointing to state-specific child shapefiles.

    The stack traces can also look like this:

    my_shapefile = fiona.open(shapefile_uri)
    File "/var/task/fiona/env.py", line 406, in wrapper
    return f(*args, **kwargs)
    File "/var/task/fiona/__init__.py", line 253, in open
    layer=layer, enabled_drivers=enabled_drivers, **kwargs)
    File "/var/task/fiona/collection.py", line 159, in __init__
    self.session.start(self, **kwargs)
    File "fiona/ogrext.pyx", line 484, in fiona.ogrext.Session.start
    File "fiona/_shim.pyx", line 80, in fiona._shim.gdal_open_vector
    fiona.errors.DriverError: Failed to read all values for 33144 records in .shx file: No such file or directory.
    

    These errors occur infrequently but are clustered in time -- see below.

    Steps to reproduce the problem.

    I'm not able to reproduce the problem consistently. My AWS lambda code opens the same S3 shapefile every time the lambda is invoked. The fiona.open call succeeds most of the time (say, >95%), but I occasionally get the errors described above (and these errors tend to be clustered in time).

    Operating system

    AWS lambda with runtime = "python3.6" (see https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html).

    Fiona and GDAL version and provenance

    I've gotten this error with both fiona==1.8.6 and fiona==1.8a2.

    opened by atorch 18
  • FionaValueError: No dataset found at path

    FionaValueError: No dataset found at path

    I suddenly keep getting:

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Python27\lib\site-packages\fiona\__init__.py", line 175, in open
        enabled_drivers=enabled_drivers)
      File "C:\Python27\lib\site-packages\fiona\collection.py", line 147, in __init__
        self.session.start(self)
      File "fiona\ogrext.pyx", line 444, in fiona.ogrext.Session.start (fiona/ogrext.c:9253)
    FionaValueError: No dataset found at path '\test.shp' using drivers: *
    

    I use fiona through rasterstats (https://github.com/perrygeo/python-rasterstats). I get this with every .shp files. Even those that used to work.

    The problem is that I don't know exactly what changed. I've updated to: Python 2.7.13 And from Gohlke: Fiona 1.7.1 GDAL 2.0.2 (because the 2.1.2 version resulted in Python crashes) From gisinternals GDAL 2.1.2 and it started not working.

    I used to use: Python 2.7.11 And from Gohlke: Fiona 1.6.3 GDAL 2.0.2 From gisinternals GDAL 2.1.0

    The problem is when I uninstall everything (even GDAL from gisinternals) and go back to 1.6.3, the problem persists. ogrext.pyx doesn't seems to be anywhere (only .pyd) but I guess that's normal?

    I apologize already for probably not giving enough information. Many thanks in advance.

    opened by vanderlist 18
  • Windows CI

    Windows CI

    A Windows CI could be useful to test Fionas behavior on windows platform or in the future to build binary wheels for windows.

    I started playing with appveyor: https://github.com/rbuffat/Fiona/blob/win/appveyor.yml, respectively: https://ci.appveyor.com/project/rbuffat/fiona @micahcochran the current status is that fiona can built but at runtime the following error is thrown:

    Traceback (most recent call last):
    File "c:\python27.10-x64\lib\site-packages\nose\suite.py", line 209, in run
        self.setUp()
    File "c:\python27.10-x64\lib\site-packages\nose\suite.py", line 292, in setUp
        self.setupContext(ancestor)
    File "c:\python27.10-x64\lib\site-packages\nose\suite.py", line 315, in setupContext
        try_run(context, names)
    File "c:\python27.10-x64\lib\site-packages\nose\util.py", line 471, in try_run
        return func()
    File "C:\projects\fiona\tests\__init__.py", line 39, in setup
        create_jsonfile(jsonfile)
    File "C:\projects\fiona\tests\__init__.py", line 20, in create_jsonfile
        import fiona
    File "build\bdist.win-amd64\egg\fiona\__init__.py", line 72, in <module>
    File "build\bdist.win-amd64\egg\fiona\collection.py", line 7, in <module>
    File "build\bdist.win-amd64\egg\fiona\ogrext.py", line 7, in <module>
    File "build\bdist.win-amd64\egg\fiona\ogrext.py", line 6, in __bootstrap__
    ImportError: DLL load failed: %1 is not a valid Win32 application.
    

    My current research revealed that this can be caused by various reasons such as missing files, missing paths, incompatible compiled dll's etc.

    intermediate 
    opened by rbuffat 18
  • remove per-feature logging to improve performance issue#685

    remove per-feature logging to improve performance issue#685

    I saw the issue #685, and so I took a stab at it based on the discussion to see if I could help move things forward.

    I commented out the debug print statements instead of deleting them so that you don't have to re-create them when you are debugging (and an official decision on what to do with them has not been reached as far as I know). However, that presents the danger of being re-introduced by accident when committing changes after debugging.

    Using this example:

    from itertools import chain, repeat
    import fiona
    def test_to_file():
        with fiona.Env(),fiona.open("../tests/data/coutwildrnp.shp") as collection:
            features = chain.from_iterable(repeat(list(collection), 2000))
            with fiona.open("/tmp/out.gpkg", "w", schema=collection.schema, crs=collection.crs, driver="GPKG") as dst:
                dst.writerecords(features)
    

    And timing it:

    %%timeit
    test_to_file()
    

    Before: 1min 3s ± 2.32 s per loop (mean ± std. dev. of 7 runs, 1 loop each) After: 15.3 s ± 1.13 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

    opened by snowman2 17
  • Support add new columns to gpkg in append mode

    Support add new columns to gpkg in append mode

    Hi, I'm using Fiona from GeoPandas, actually there is the append mode, but if the data we want to append has columns that does not exists will fails. I'm constructing a big gpkg file but I don't have a way to know all the columns beforehand.

      File "fiona/ogrext.pyx", line 1280, in fiona.ogrext.WritingSession.writerecs
    ValueError: Record does not match collection schema: dict_keys(list_with_column_names)
    

    Thx!

    opened by latot 0
  • Update and streamline the readme

    Update and streamline the readme

    I've made the readme fresher, removing mentions of obsolete GDAL installers, and much shorter, removing some old examples and details about installation from source that are better off in a more advanced doc.

    Updating the readme example suggested that Feature &c should be exported in fiona/__init__.py and that the Geometry class should know about __geo_interface__ to integrate better with Shapely.

    opened by sgillies 0
  • Fail to read an online zipped ESRI GDB file with Fiona 1.8.22, but success with Fiona 1.8.13

    Fail to read an online zipped ESRI GDB file with Fiona 1.8.22, but success with Fiona 1.8.13

    I expected to read an online zipped ESRI GDB file with Fiona 1.8.22, it failed, while when I try it in another environment with Fiona 1.8.13, it succeeded.

    Steps to reproduce the problem.

    import fiona import geopandas as gp path="https://raw.githubusercontent.com/UrbanGISer/Test/main/Case05new.gdb.zip" c = fiona.open(path,layer=0) gdf = gp.GeoDataFrame.from_features(c) gdf.crs=c.crs.get('init') gdf.head()

    Operating system

    Win 11

    Fiona and GDAL version and provenance

    Fiona 1.8.22 with print (osgeo.gdal.version) 3.5.2 Fiona 1.8.13 with print (osgeo.gdal.version) 3.0.2

    needs discussion 
    opened by UrbanGISer 3
  • Update README.rst with infos regarding to GDAL embedded wheels for windows

    Update README.rst with infos regarding to GDAL embedded wheels for windows

    Expected behavior and actual behavior.

    No bug here, but an out of date documentation (see. discussion on fiona.groups.io).

    The actual documentation doesn't acknowledge that there are now windows wheels on pypi and that they include a GDAL shared library.

    The documentation might also benefit from a paragraph stating what could and what couldn't be done regarding to diverse GDAL's bindings (osgeo and all it's dependencies).

    bug docs 
    opened by tgrandje 0
  • test_no_append_driver_cannot_append fails

    test_no_append_driver_cannot_append fails

    https://gist.github.com/risicle/2631e23beaf492e95d6d1ef44d3d789e

    Operating system

    NixOS 22.11

    Fiona and GDAL version and provenance

    Fiona 1.8.22 installed via Nix GDAL 3.6.0.1 installed via Nix

    opened by dotlambda 1
Releases(1.9b1)
  • 1.9b1(Dec 13, 2022)

    The fist 1.9 beta release is ready for early testing. All of the features planned for 1.9.0 are in this release.

    The binary wheels on PyPI include GDAL 3.5.3, GEOS 3.11.1, and PROJ 9.0.1. To try the wheels, run pip install --pre fiona.

    New features

    • Add listdir and listlayers method to io.MemoryFile (resolving #754).
    • Add support for TIN and triangle geometries (#1163).
    • Add an allow_unsupported_drivers option to fiona.open() (#1126).
    • Added support for the OGR StringList field type (#1141).

    Changes and bug fixes

    • Missing and unused imports have been added or removed.
    • Make sure that errors aren't lost when a collection can't be saved properly (#1169).
    • Ensure that ZipMemoryFile have the proper GDAL name after creation so that we can use listdir() (#1092).
    • The fiona._loading module, which supports DLL loading on Windows, has been moved into __init__.py and is no longer used anywhere else (#1168).
    • Move project metadata to pyproject.toml (#1165).
    • Update drvsupport.py to reflect new format capabilities in GDAL 3.6.0 (#1122).
    • Remove debug logging from env and _env modules.
    Source code(tar.gz)
    Source code(zip)
  • 1.9a3(Oct 18, 2022)

    Builds now require Cython >= 0.29.29 because of https://github.com/cython/cython/issues/4609 (see https://github.com/Toblerity/Fiona/issues/1143).

    Wheels include GDAL 3.5.2, PROJ 9.0.1, and GEOS 3.11.0 and are available for Python versions 3.7-3.11.

    Source code(tar.gz)
    Source code(zip)
  • 1.8.22(Oct 14, 2022)

    Builds now require Cython >= 0.29.29 because of https://github.com/cython/cython/issues/4609 (see #1143).

    Wheels include GDAL 3.4.3, PROJ 8.2.1, and GEOS 3.10.2.

    Source code(tar.gz)
    Source code(zip)
  • 1.9a2(Jun 10, 2022)

    The second 1.9 pre-release is ready for early testing. Not all of the features planned for 1.9.0 are in this release, and some of the features here may yet be modified. Please pay close attention to the deprecations and packaging changes.

    The binary wheels on PyPI include GDAL 3.5.0 and PROJ 9.0.0, the latest versions of each. To try the wheels, run pip install --pre fiona.

    Deprecations:

    • Fiona's API methods will accept feature and geometry dicts in 1.9.0, but this usage is deprecated. Instances of Feature and Geometry will be required in 2.0.
    • The precision keyword argument of fiona.transform.transform_geom is deprecated and will be removed in version 2.0.
    • Deprecated usage has been eliminated in the project. Fiona's tests pass when run with a -Werror::DeprecationWarning filter.

    Changes:

    • Fiona's FionaDeprecationWarning now sub-classes DeprecationWarning.
    • Some test modules have beeen re-formatted using black.

    New features:

    • Fiona Collections now carry a context exit stack into which we can push fiona Envs and MemoryFiles (#1059).
    • Fiona has a new CRS class, like rasterio's, which is compatible with the CRS dicts of previous versions (#714).
    Source code(tar.gz)
    Source code(zip)
  • 1.9a1(May 19, 2022)

    The first 1.9 pre-release is ready for early testing. Not all of the features planned for 1.9.0 are in this release, and some of the features here may yet be modified. Please pay close attention to the deprecations and packaging changes.

    The binary wheels on PyPI include GDAL 3.5.0 and PROJ 9.0.0, the latest versions of each.

    Deprecations:

    • The fiona.drivers() function has been deprecated and will be removed in version 2.0. It should be replaced by fiona.Env().
    • The new fiona.meta module will be renamed to fiona.drivers in version 2.0.

    Packaging:

    • Source distributions contain no C source files and require Cython to create them from .pyx files (#1096).

    Changes:

    • Shims for various versions of GDAL have been removed and are replaced by Cython compilation conditions (#1093).
    • Use of CURL_CA_BUNDLE environment variable is replaced by a more specific GDAL/PROJ_CURL_CA_BUNDLE (#1095).
    • Fiona's feature accessors now return instances of fiona.model.Feature instead of Python dicts (#787). The Feature class is compatible with code that expects GeoJSON-like dicts but also provides id, geometry, and properties attributes. The last two of these are instances of fiona.model.Geometry and fiona.model.Properties.
    • GDAL 3.1.0 is the minimum GDAL version.
    • Drop Python 2, and establish Python 3.7 as the minimum version (#1079).
    • Remove six and reduce footprint of fiona.compat (#985).

    New features:

    • The appropriate format driver can be detected from filename in write mode (#948).
    • Driver metadata including dataset open and dataset and layer creations options are now exposed through methods of the fiona.meta module (#950).
    • CRS WKT format support (#979).
    • Add 'where' SQL clause to set attribute filter (#961, #1097).

    Bug fixes:

    • Env and Session classes have been updated for parity with rasterio and to resolve a credential refresh bug (#1055).
    Source code(tar.gz)
    Source code(zip)
  • 1.8.21(Feb 7, 2022)

    Changes:

    • Driver mode support tests have been made more general and less susceptible to driver quirks involving feature fields and coordinate values (#1060).
    • OSError is raised on attempts to open a dataset in a Python file object in "a" mode (see #1027).
    • Upgrade attrs, cython, etc to open up Python 3.10 support (#1049).

    Bug fixes:

    • Allow FieldSkipLogFilter to handle exception messages as well as strings (reported in #1035).
    • Clean up VSI files left by MemoryFileBase, resolving #1041.
    • Hard-coded "utf-8" collection encoding added in #423 has been removed (#1057).
    Source code(tar.gz)
    Source code(zip)
  • 1.8.13.post1(Feb 22, 2020)

    This release is being made to improve binary wheel compatibility with shapely 1.7.0. There have been no changes to the fiona package code since 1.8.13.

    Source code(tar.gz)
    Source code(zip)
  • 1.8.0(Oct 31, 2018)

    Fiona 1.8.0 is on PyPI today. Congratulations to all 46 developers (see the credits) file and many thanks to everyone who took the time to report a bug or test a new feature.

    Much of the motivation for this version has been provided by the GeoPandas project. Working with Joris Van den Bossche et al. on various issues has been a pleasure.

    There are no known breaking changes in 1.8.0. Python warnings should be expected in several cases of class and method deprecation.

    • The fiona.drivers() context manager is being replaced by fiona.Env(), which also registers format drivers and has the same properties as the GDAL configuration manager in Rasterio.
    • Collection slicing will be disallowed in a future version of Fiona to remove the confusion between mapping and list semantics for Collection objects. Code such as fiona.open(“example.shp”)[1:10] should be changed to list(fiona.open(“example.shp”))[1:10].

    This version has new features, including a set from the Rasterio project.

    • Fiona has an increased and configurable transaction size for record writes, which makes the GeoPackage format fully usable in Fiona.
    • The “http” and “https” URI schemes for datasets are now fully supported, providing direct access to vector data on the web. Support for an “s3” does the same for data in AWS S3 buckets.
    • Support for “zip”, “zip+https”, and “zip+s3” datasets allows users to access records in zipped Shapefiles (for example) without needing to unzip them, whether on a local file system, on the web, or in S3.
    • New MemoryFile and ZipMemoryFile classes provide easy access to datasets in streams of bytes.

    Major refactoring was required to bring new features over from Rasterio and to modernize our use of Cython. This was a huge lift, largely done by Joshua Arnott. Elliott Sales de Andrade took the lead on finishing the migration of Fiona’s tests to pytest.

    Source code(tar.gz)
    Source code(zip)
Blender addons to make the bridge between Blender and geographic data

Blender GIS Blender minimal version : 2.8 Mac users warning : currently the addon does not work on Mac with Blender 2.80 to 2.82. Please do not report

null 5.9k Jan 2, 2023
Python tools for geographic data

GeoPandas Python tools for geographic data Introduction GeoPandas is a project to add support for geographic data to pandas objects. It currently impl

GeoPandas 3.5k Jan 3, 2023
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

django-rest-framework-gis Geographic add-ons for Django Rest Framework - Mailing List. Install last stable version from pypi pip install djangorestfra

OpenWISP 981 Jan 3, 2023
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

OpenWISP 982 Jan 6, 2023
Mmdb-server - An open source fast API server to lookup IP addresses for their geographic location

mmdb-server mmdb-server is an open source fast API server to lookup IP addresses

Alexandre Dulaunoy 67 Nov 25, 2022
This tool reads the route of a hike and generates a table of iNaturalist observations along the trails.

This tool reads the route of a hike and generates a table of iNaturalist observations along the trails. It also shows the observations and the route of the hike on a map. Moreover, it saves waypoints of the iNaturalist observations for offline navigation with a GPS device or smartphone.

null 1 Nov 21, 2021
Using Global fishing watch's data to build a machine learning model that can identify illegal fishing and poaching activities through satellite and geo-location data.

Using Global fishing watch's data to build a machine learning model that can identify illegal fishing and poaching activities through satellite and geo-location data.

Ayush Mishra 3 May 6, 2022
A Django application that provides country choices for use with forms, flag icons static files, and a country field for models.

Django Countries A Django application that provides country choices for use with forms, flag icons static files, and a country field for models. Insta

Chris Beaven 1.2k Jan 3, 2023
Ingest and query genomic intervals from multiple BED files

Ingest and query genomic intervals from multiple BED files.

null 4 May 29, 2021
🌐 Local tile server for viewing geospatial raster files with ipyleaflet

?? Local Tile Server for Geospatial Rasters Need to visualize a rather large raster (gigabytes) you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 4, 2023
🌐 Local tile server for viewing geospatial raster files with ipyleaflet or folium

?? Local Tile Server for Geospatial Rasters Need to visualize a rather large (gigabytes) raster you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 4, 2023
Create Siege configuration files from Cloud Optimized GeoTIFF.

cogeo-siege Documentation: Source Code: https://github.com/developmentseed/cogeo-siege Description Create siege configuration files from Cloud Optimiz

Development Seed 3 Dec 1, 2022
Open Data Cube analyses continental scale Earth Observation data through time

Open Data Cube Core Overview The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth obse

Open Data Cube 410 Dec 13, 2022
A public data repository for datasets created from TransLink GTFS data.

TransLink Spatial Data What: TransLink is the statutory public transit authority for the Metro Vancouver region. This GitHub repository is a collectio

Henry Tang 3 Jan 14, 2022
OSMnx: Python for street networks. Retrieve, model, analyze, and visualize street networks and other spatial data from OpenStreetMap.

OSMnx OSMnx is a Python package that lets you download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street

Geoff Boeing 4k Jan 8, 2023
python toolbox for visualizing geographical data and making maps

geoplotlib is a python toolbox for visualizing geographical data and making maps data = read_csv('data/bus.csv') geoplotlib.dot(data) geoplotlib.show(

Andrea Cuttone 976 Dec 11, 2022
Tool to suck data from ArcGIS Server and spit it into PostgreSQL

chupaESRI About ChupaESRI is a Python module/command line tool to extract features from ArcGIS Server map services. Name? Think "chupacabra" or "Chupa

John Reiser 34 Dec 4, 2022
Processing and interpolating spatial data with a twist of machine learning

Documentation | Documentation (dev version) | Contact | Part of the Fatiando a Terra project About Verde is a Python library for processing spatial da

Fatiando a Terra 468 Dec 20, 2022
gpdvega is a bridge between GeoPandas and Altair that allows to seamlessly chart geospatial data

gpdvega gpdvega is a bridge between GeoPandas a geospatial extension of Pandas and the declarative statistical visualization library Altair, which all

Ilia Timofeev 49 Jul 25, 2022