The openspoor package is intended to allow easy transformation between different geographical and topological systems commonly used in Dutch Railway

Overview

Openspoor

alt text

The openspoor package is intended to allow easy transformation between different geographical and topological systems commonly used in Dutch Railway. Its goal is to be publicly available and function as an open source package.

Currently the openspoor package allows the following transformations:

Type of input:

  • Point data

These transformations can be performed between the following systems:

Geographical systems:

  • WGS84 coordinate system (commonly known as GPS coordinates)
  • EPSG:28992 coordinate system (commonly known in the Netherlands as Rijksdriehoek)

Topological systems:

  • Geocode and geocode kilometrering
  • Spoortak and spoortak kilometrering (unavailable on switches)

Getting Started

Installation

Installation using anaconda

  • Clone the "openspoor" repository
    • pip install openspoor
  • create an environment:
    • conda create -n openspoorenv python==3.6.12
  • activate the environment:
    • conda activate openspoorenv
  • If you are installing on Windows OS with Anaconda, first install rtree and geopandas through anaconda with the commands:
    • conda install rtree==0.8.3 -y
    • conda install geopandas==0.6.1 -y
  • In the root directory of the repository, execute the command:
    • pip install -r requirements.txt
  • In the root directory of the repository, execute the command:
    • pip install .
  • In the root directory of the repository, execute the command:
    • python -m pytest
  • If all the test succeed, the openspoor package is ready to use and you are on the right "track"!

Demonstration notebook

In the future a notebook will be added that demonstrates the use of the openspoor package. For now one can take the code in the acceptance tests as example of how to use the package.

Dependencies

The transformations available in the openspoor package rely completely on data and API's made available at https://mapservices.prorail.nl/. Be aware of this dependency and specifically of the following possible issues:

  • The use of API's on mapservices.prorail.nl is changed
  • The output data of the mapservices API's is changed (with added, removed or missing columns for instance)

Furthermore mapservices.prorail.nl only provides current information about the topological systems used in Dutch Railways. As the topological systems tend to change with time, due to changing infrastructure and naming conventions, the current topological system is not necessarily sufficient to provide transformations on historical data. In the future we hope to add historical topological systems as part of the functionality of this package in such a way that it is available publicly.

Structure

The structure of the openspoor package is largely split in two categories.

MapservicesData

The MapservicesData classes use mapservices.prorail.nl API's to retrieve the necessary data to perform transformations. The essentially function as an interface with the topological systems used by ProRail.

  • PUICMapservices provides general data about railway tracks (spoor) and switches (wissel and kruisingbenen). This contains information regarding Geocode, geocodekilometrering, but also Spoortak identificatie.
  • SpoortakMapservices provides information about railway tracks concerning Spoortak identificatie and lokale kilometrering.

Transformers

The various transformers use the geopandas dataframes obtained by MapservicesData objects to add additional geographical or topological systems to a given geopandas input dataframe. The current transformers only function for geopandas dataframes containing Point data. The available transformers are:

  • TransformerCoordinatesToSpoor: transforms WGS84 or EPSG:28992 coordinates to spoortak and lokale kilomtrering as well as geocode and geocode kilometrering.
  • TransformerGeocodeToCoordinates: transforms geocode and geocode kilometrering to WGS84 or EPSG:28992 coordinates.
  • TransformerSpoorToCoordinates: transforms spoortak and lokale kilometrering to WGS84 or EPSG:28992 coordinates.

Release History

  • 0.1.0
    • The first proper release
    • ADD: transform point data between geographical systems.
  • 0.0.1
    • Work in progress

Contributing

The openspoor package stimulates every other person the contribute to the package. To do so:

  • Fork it
  • Create your feature branch (git checkout -b feature/fooBar)
  • Commit your changes (git commit -am 'Add some fooBar')
  • Push to the branch (git push origin feature/fooBar)
  • Create a new Pull Request with 3 obligated reviewers from the developement team.

You could also contribute by thinking of possible new features. The current backlog is:

  • Make the package available for the "spoor" industry.
Comments
  • Kleine bugfix: drop duplicates makes sure that merge returns same size df as input df

    Kleine bugfix: drop duplicates makes sure that merge returns same size df as input df

    Tijdens het gebruik van TransformerGeocodeToCoordinates zag ik dat df_with_xy.index = index fout ging omdat de df_with_xy groter was geworden dan de input df. Dit komt omdat er duplicaten in xy_information zitten (geen idee waarom), maar dit is dus gefixt met een drop_duplicates.

    opened by OscarOnTrack 1
  • fix pyproj version

    fix pyproj version

    pyproj < 3 did not work for Apple devices (with M1 chip), and the package works on Windows without this requirement. Furthermore, the <3 version requirement in setup.py is inconsistent with the 3.3.* requirements in requirements.txt, so it makes sense to remove the <3 requirement in setup.py

    opened by OscarOnTrack 1
  • Generic API call

    Generic API call

    Make the URL in SpoortakMapServices configurable to allow it to read multiple layers. Can be combined wihth FeatureServerOverview for added user simplicity

    opened by JoostProRail 1
  • Spoortakmodel

    Spoortakmodel

    Added a mapping and inspection classes to work with spoortak model version 3 up to and including version 17

    Note the spoortak models data has not yet been included. We are verifying if they can be distributed or not trough github. Thus the integration tests will fail due to missing data.

    opened by Nozziel 1
  • Make plottable objects more consistent

    Make plottable objects more consistent

    In particular:

    • Base all 3 objects on a GeoDataFrame under self.data. PlottingPoints will require the most work here.
    • Fix the popup argument in a consistent manner (none, str or tuple of strings)
      • Fix the data argument in a consistent manner (string or geopandas geodataframe)
    • General support for self.color?

    Additionally, if possible we can try to make PlottingPoints work for pandas dataframes too.

    opened by JoostProRail 0
  • Update readme and requirements

    Update readme and requirements

    This started as an upgrade to the version number, but also upgraded the readme and added a requirement. The setup instructions were also verified on a Windows machine and are working now.

    opened by JoostProRail 0
  • Make secure requests to mapservices

    Make secure requests to mapservices

    Making requests results in a lot of warnings:

    InsecureRequestWarning: Unverified HTTPS request is being made to host 'mapservices.prorail.nl'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning,

    opened by JoostProRail 0
  • Refactor SpoorKaart

    Refactor SpoorKaart

    Set up spoorkaarten outside of a context manager. Suggested approach would be to make the below workable.

    map = SpoorKaart() map.add(plottable object) map.add(plottable object2) map.show()

    map.save('save_location.html')

    opened by JoostProRail 0
  • fix windows install trough conda

    fix windows install trough conda

    loguru 0.4.0 is not available on conda-forge only 0.4.1 old rtree will not properly install on windows, needed an updated version

    simplified the readme to just conda install with conda-forge as extra channel. This will now work in windows as well as mac/unix

    Switched to Python 3.8 in the readme (because I could, all the unit tests were still green)

    opened by Nozziel 0
  • Add feature server finding utilities

    Add feature server finding utilities

    Is dit iets wat we willen? Zelf vond ik dit een handige richting om ook andere features eruit te kunnen halen. Mochten we openspoor daar voor klaar willen maken dan moeten we nog regelen dat de URL's die je hier vindt gebruikt kunnen worden in latere queries: de meeste logica daarvoor bestaat al, het moet alleen een stukje generieker dan. Zomaar een idee, suggesties zijn welkom!

    opened by JoostProRail 0
  • Check requirements versions

    Check requirements versions

    Openspoor seems to be incompatible with more recent versions of numpy (1.22 and above). This is further complicated by the fact that GDAL and Fiona are installed through conda instead of Pip. More futureproof versions (or installation procedures) would be helpful to keep up with recent developments.

    opened by JoostProRail 2
  • Wrong interpolation in TransformerSpoortakToCoordinates.py

    Wrong interpolation in TransformerSpoortakToCoordinates.py

    The interpolation in the transform method in TransformerSpoortakToCoordinates.py is not valid. At the start of the geometry the lokale km is not (always) zero. So it should use the travelled distance on that geometry (difference in lokale_km from start and the point of interest). But that would still not be a correct way to do it because than it assumes that a geocode km is linear, which is not the case. We should use M-values for this.

    opened by Neliz 0
  • Update geocode conversion

    Update geocode conversion

    There now exists a layer with both spoortakken and geocode information in it; this can be used as a replacement for the current xy_to_geocode api calls. Gathering this layer and interpolating based on the information in here would therefore be a more efficient way to get geocodes for a lot of coordinates. Some effort in this direction was already done by the beeldherkenning configuratie team. They pointed out that one fix would be is to use the geocode information (stored in the M (measurement) value in the featureserver can overwrite the Z-value in the data (which is not that useful), from where it can be used to interpolate along the linestrings.

    https://mapservices.prorail.nl/arcgis/rest/services/Geleidingssysteem_010/FeatureServer/34

    opened by JoostProRail 0
  • Fix PUICMapservices

    Fix PUICMapservices

    transformer PUICMapservices does not work with the most recent featureservers anymore. It was designed with the layers as described below with original_* in mind; the new_* versions are the latest versions of these layers.

    new_wisselbeen = MapServicesQuery(puic_mapservices.wisselkruisingbeen_query.layer_url.values[0]).load_data() original_wisselbeen = MapServicesQuery("https://mapservices.prorail.nl/arcgis/rest/services/Geleidingssysteem_007/FeatureServer/12/").load_data() print(new_wisselbeen.shape, original_wisselbeen.shape)

    new_spoortakdeel = MapServicesQuery(puic_mapservices.wisselkruisingbeen_query.layer_url.values[0]).load_data() original_spoortakdeel = MapServicesQuery("https://mapservices.prorail.nl/arcgis/rest/services/Geleidingssysteem_007/FeatureServer/13/").load_data() print(new_spoortakdeel.shape, original_spoortakdeel.shape)

    opened by JoostProRail 0
  • pyproj package >= 3 does not work, will be a problem in the long run (e.g. M1 chip macbook)

    pyproj package >= 3 does not work, will be a problem in the long run (e.g. M1 chip macbook)

    Joost and Oscar have tried to get openspoor working with versions >= 3, but one acceptance test does not fully work then (test_acceptance_TransformerCoordinatesToSpoor). Most points are projected as expected, but 2 are not (of which 1 cannot be fixed with a larger buffer distance). This is strange, since a point just 4 meters next to it can be projected successfully.

    Reason for fixing: pyproj < 3 does not work on M1 chip (macbook)

    opened by OscarOnTrack 2
Owner
null
BRepNet: A topological message passing system for solid models

BRepNet: A topological message passing system for solid models This repository contains the an implementation of BRepNet: A topological message passin

Autodesk AI Lab 42 Dec 30, 2022
OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.

OpenFace 2.2.0: a facial behavior analysis toolkit Over the past few years, there has been an increased interest in automatic facial behavior analysis

Tadas Baltrusaitis 5.8k Dec 31, 2022
City-seeds - A random generator of cultural characteristics intended to spark ideas and help draw threads

City Seeds This is a random generator of cultural characteristics intended to sp

Aydin O'Leary 2 Mar 12, 2022
MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

Microsoft 5.7k Jan 9, 2023
A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Awesome Pretrained StyleGAN2 A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution. Note the readme is a

Justin 1.1k Dec 24, 2022
Code for Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022)

Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022) We consider how a user of a web servi

joisino 20 Aug 21, 2022
Data manipulation and transformation for audio signal processing, powered by PyTorch

torchaudio: an audio library for PyTorch The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the

null 1.9k Dec 28, 2022
GraphGT: Machine Learning Datasets for Graph Generation and Transformation

GraphGT: Machine Learning Datasets for Graph Generation and Transformation Dataset Website | Paper Installation Using pip To install the core environm

y6q9 50 Aug 18, 2022
DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

Facebook Research 145 Dec 30, 2022
Efficient Householder transformation in PyTorch

Efficient Householder Transformation in PyTorch This repository implements the Householder transformation algorithm for calculating orthogonal matrice

Anton Obukhov 49 Nov 20, 2022
Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model The task of age transformation illustrates the change of an individual

null 444 Dec 30, 2022
git《Tangent Space Backpropogation for 3D Transformation Groups》(CVPR 2021) GitHub:1]

LieTorch: Tangent Space Backpropagation Introduction The LieTorch library generalizes PyTorch to 3D transformation groups. Just as torch.Tensor is a m

Princeton Vision & Learning Lab 482 Jan 6, 2023
Meta Representation Transformation for Low-resource Cross-lingual Learning

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning This repo hosts the code for MetaXL, published at NAACL 2021. [Meta

Microsoft 36 Aug 17, 2022
[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration This repository contains the implementation of our paper Locally Aware Pi

sfwang 70 Dec 19, 2022
Improving Contrastive Learning by Visualizing Feature Transformation, ICCV 2021 Oral

Improving Contrastive Learning by Visualizing Feature Transformation This project hosts the codes, models and visualization tools for the paper: Impro

Bingchen Zhao 83 Dec 15, 2022
[CVPR'21] Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation

Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Y

null 118 Dec 26, 2022
Fuzzing tool (TFuzz): a fuzzing tool based on program transformation

T-Fuzz T-Fuzz consists of 2 components: Fuzzing tool (TFuzz): a fuzzing tool based on program transformation Crash Analyzer (CrashAnalyzer): a tool th

HexHive 244 Nov 9, 2022
Affine / perspective transformation in Pose Estimation with Tensorflow 2

Pose Transformation Affine / Perspective transformation in Pose Estimation with Tensorflow 2 Introduction 이 repo는 pose estimation을 연구하고 개발하는 데 도움이 되기

Kim Junho 1 Dec 22, 2021
Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

CoulombGas This code implements the neural canonical transformation approach to the thermodynamic properties of uniform electron gas. Building on JAX,

FermiFlow 9 Mar 3, 2022