A Python validator for SHACL

Overview

pySHACL

A Python validator for SHACL.

Build Status

DOI Downloads Downloads Downloads

This is a pure Python module which allows for the validation of RDF graphs against Shapes Constraint Language (SHACL) graphs. This module uses the rdflib Python library for working with RDF and is dependent on the OWL-RL Python module for OWL2 RL Profile based expansion of data graphs.

This module is developed to adhere to the SHACL Recommendation:

Holger Knublauch; Dimitris Kontokostas. Shapes Constraint Language (SHACL). 20 July 2017. W3C Recommendation. URL: https://www.w3.org/TR/shacl/ ED: https://w3c.github.io/data-shapes/shacl/

Community for Help and Support

The SHACL community has a discord server for discussion of topics around SHACL and the SHACL specification.

Use this invitation link: https://discord.gg/RTbGfJqdKB to join the server

There is a #pyshacl channel in which discussion around this python library can held, and you can ask for general pyshacl help too.

Installation

Install with PIP (Using the Python3 pip installer pip3)

$ pip3 install pyshacl

Or in a python virtualenv (these example commandline instructions are for a Linux/Unix based OS)

$ python3 -m virtualenv --python=python3 --no-site-packages .venv
$ source ./.venv/bin/activate
$ pip3 install pyshacl

To exit the virtual enviornment:

$ deactivate

Command Line Use

For command line use: (these example commandline instructions are for a Linux/Unix based OS)

$ pyshacl -s /path/to/shapesGraph.ttl -m -i rdfs -a -j -f human /path/to/dataGraph.ttl

Where

  • -s is an (optional) path to the shapes graph to use
  • -e is an (optional) path to an extra ontology graph to import
  • -i is the pre-inferencing option
  • -f is the ValidationReport output format (human = human-readable validation report)
  • -m enable the meta-shacl feature
  • -a enable SHACL Advanced Features
  • -j enable SHACL-JS Features (if pyhsacl[js] is installed)

System exit codes are: 0 = DataGraph is Conformant 1 = DataGraph is Non-Conformant 2 = The validator encountered a RuntimeError (check stderr output for details) 3 = Not-Implemented; The validator encountered a SHACL feature that is not yet implemented.

Full CLI Usage options:

$ pyshacl -h
$ python3 -m pyshacl -h
usage: pyshacl [-h] [-s [SHACL]] [-e [ONT]] [-i {none,rdfs,owlrl,both}] [-m]
               [--imports] [--abort] [-a] [-j] [-d] [-f {human,table,turtle,xml,json-ld,nt,n3}]
               [-df {auto,turtle,xml,json-ld,nt,n3}]
               [-sf {auto,turtle,xml,json-ld,nt,n3}]
               [-ef {auto,turtle,xml,json-ld,nt,n3}] [-V] [-o [OUTPUT]]
               DataGraph

Run the pySHACL validator from the command line.

positional arguments:
  DataGraph             The file containing the Target Data Graph.

optional arguments:
  -h, --help            show this help message and exit
  -s [SHACL], --shacl [SHACL]
                        A file containing the SHACL Shapes Graph.
  -e [ONT], --ont-graph [ONT]
                        A file path or URL to a document containing extra
                        ontological information to mix into the data graph.
  -i {none,rdfs,owlrl,both}, --inference {none,rdfs,owlrl,both}
                        Choose a type of inferencing to run against the Data
                        Graph before validating.
  -m, --metashacl       Validate the SHACL Shapes graph against the shacl-
                        shacl Shapes Graph before validating the Data Graph.
  --imports             Allow import of sub-graphs defined in statements with
                        owl:imports.
  -a, --advanced        Enable support for SHACL Advanced Features.
  -j, --js              Enable support for SHACL-JS Features.
  -it, --iterate-rules  Interate SHACL Rules until steady state is found (only available in Advanced Mode)
  --abort               Abort on first error.
  -d, --debug           Output additional runtime messages, including violations that didn\'t
                        lead to non-conformance.
  -f {human,turtle,xml,json-ld,nt,n3}, --format {human,turtle,xml,json-ld,nt,n3}
                        Choose an output format. Default is "human".
  -df {auto,turtle,xml,json-ld,nt,n3}, --data-file-format {auto,turtle,xml,json-ld,nt,n3}
                        Explicitly state the RDF File format of the input
                        DataGraph file. Default="auto".
  -sf {auto,turtle,xml,json-ld,nt,n3}, --shacl-file-format {auto,turtle,xml,json-ld,nt,n3}
                        Explicitly state the RDF File format of the input
                        SHACL file. Default="auto".
  -ef {auto,turtle,xml,json-ld,nt,n3}, --ont-file-format {auto,turtle,xml,json-ld,nt,n3}
                        Explicitly state the RDF File format of the extra
                        ontology file. Default="auto".
  -V, --version         Print the PySHACL version and exit.
  -o [OUTPUT], --output [OUTPUT]
                        Send output to a file (defaults to stdout).

Python Module Use

For basic use of this module, you can just call the validate function of the pyshacl module like this:

from pyshacl import validate
r = validate(data_graph,
      shacl_graph=sg,
      ont_graph=og,
      inference='rdfs',
      abort_on_first=False,
      allow_warnings=False,
      meta_shacl=False,
      advanced=False,
      js=False,
      debug=False)
conforms, results_graph, results_text = r

Where:

  • data_graph is an rdflib Graph object or file path of the graph to be validated
  • shacl_graph is an rdflib Graph object or file path or Web URL of the graph containing the SHACL shapes to validate with, or None if the SHACL shapes are included in the data_graph.
  • ont_graph is an rdflib Graph object or file path or Web URL a graph containing extra ontological information, or None if not required.
  • inference is a Python string value to indicate whether or not to perform OWL inferencing expansion of the data_graph before validation. Options are 'rdfs', 'owlrl', 'both', or 'none'. The default is 'none'.
  • abort_on_first (optional) bool value to indicate whether or not the program should abort after encountering the first validation failure or to continue. Default is to continue.
  • allow_warnings (optional) bool value, Shapes marked with severity of Warning or Info will not cause result to be invalid.
  • meta_shacl (optional) bool value to indicate whether or not the program should enable the Meta-SHACL feature. Default is False.
  • advanced: (optional) bool value to enable SHACL Advanced Features
  • js: (optional) bool value to enable SHACL-JS Features (if pyshacl[js] is installed)
  • debug (optional) bool value to indicate whether or not the program should emit debugging output text, including violations that didn't lead to non-conformance overall. So when debug is True don't judge conformance by absense of violation messages. Default is False.

Some other optional keyword variables available on the validate function:

  • data_graph_format: Override the format detection for the given data graph source file.
  • shacl_graph_format: Override the format detection for the given shacl graph source file.
  • ont_graph_format: Override the format detection for the given extra ontology graph source file.
  • iterate_rules: Interate SHACL Rules until steady state is found (only works with advanced mode).
  • do_owl_imports: Enable the feature to allow the import of subgraphs using owl:imports for the shapes graph and the ontology graph. Note, you explicitly cannot use this on the target data graph.
  • serialize_report_graph: Convert the report results_graph into a serialised representation (for example, 'turtle')
  • check_dash_result: Check the validation result against the given expected DASH test suite result.
  • check_sht_result: Check the validation result against the given expected SHT test suite result.

Return value:

  • a three-component tuple containing:
    • conforms: a bool, indicating whether or not the data_graph conforms to the shacl_graph
    • results_graph: a Graph object built according to the SHACL specification's Validation Report structure
    • results_text: python string representing a verbose textual representation of the Validation Report

Python Module Call

You can get an equivalent of the Command Line Tool using the Python3 executable by doing:

$ python3 -m pyshacl

Errors

Under certain circumstances pySHACL can produce a Validation Failure. This is a formal error defined by the SHACL specification and is required to be produced as a result of specific conditions within the SHACL graph. If the validator produces a Validation Failure, the results_graph variable returned by the validate() function will be an instance of ValidationFailure. See the message attribute on that instance to get more information about the validation failure.

Other errors the validator can generate:

  • ShapeLoadError: This error is thrown when a SHACL Shape in the SHACL graph is in an invalid state and cannot be loaded into the validation engine.
  • ConstraintLoadError: This error is thrown when a SHACL Constraint Component is in an invalid state and cannot be loaded into the validation engine.
  • ReportableRuntimeError: An error occurred for a different reason, and the reason should be communicated back to the user of the validator.
  • RuntimeError: The validator encountered a situation that caused it to throw an error, but the reason does concern the user.

Unlike ValidationFailure, these errors are not passed back as a result by the validate() function, but thrown as exceptions by the validation engine and must be caught in a try ... except block. In the case of ShapeLoadError and ConstraintLoadError, see the str() string representation of the exception instance for the error message along with a link to the relevant section in the SHACL spec document.

Windows CLI

Pyinstaller can be used to create an executable for Windows that has the same characteristics as the Linux/Mac CLI program. The necessary .spec file is already included in pyshacl/pyshacl-cli.spec. The pyshacl-cli.spec PyInstaller spec file creates a .exe for the pySHACL Command Line utility. See above for the pySHACL command line util usage instructions.

See the PyInstaller installation guide for info on how to install PyInstaller for Windows.

Once you have pyinstaller, use pyinstaller to generate the pyshacl.exe CLI file like so:

$ cd src/pyshacl
$ pyinstaller pyshacl-cli.spec

This will output pyshacl.exe in the dist directory in src/pyshacl.

You can now run the pySHACL Command Line utility via pyshacl.exe. See above for the pySHACL command line util usage instructions.

Compatibility

PySHACL is a Python3 library. For best compatibility use Python v3.7 or greater. Python3 v3.6 or below is not supported and this library does not work on Python v2.7.x or below.

PySHACL is now a PEP518 & PEP517 project, it uses pyproject.toml and poetry to manage dependencies, build and install.

For best compatibility when installing from PyPI with pip, upgrade to pip v18.1.0 or above.

  • If you're on Ubuntu 16.04 or 18.04, you will need to run sudo pip3 install --upgrade pip to get the newer version.

Features

A features matrix is kept in the FEATURES file.

Changelog

A comprehensive changelog is kept in the CHANGELOG file.

Benchmarks

This project includes a script to measure the difference in performance of validating the same source graph that has been inferenced using each of the four different inferencing options. Run it on your computer to see how fast the validator operates for you.

License

This repository is licensed under Apache License, Version 2.0. See the LICENSE deed for details.

Contributors

See the CONTRIBUTORS file.

Citation

DOI: 10.5281/zenodo.4750840 (For all versions/latest version)

Contacts

Project Lead: Nicholas Car Senior Experimental Scientist CSIRO Land & Water, Environmental Informatics Group Brisbane, Qld, Australia [email protected] http://orcid.org/0000-0002-8742-7730

Lead Developer: Ashley Sommer Informatics Software Engineer CSIRO Land & Water, Environmental Informatics Group Brisbane, Qld, Australia [email protected] https://orcid.org/0000-0003-0590-0131

Comments
  • Access to inferred triples

    Access to inferred triples

    According to 8.4 General Execution Instructions for SHACL Rules implementations modify the data graph if triples get inferred, and/or may "construct a logical data graph that has the original data as one subgraph and a dedicated inferences graph as another subgraph, and where the inferred triples get added to the inferences graph only."

    I've been following the Classification With SHACL Rules article and I would like to extract the graph of inferred triples which would include <http://bakery.com/ns#AppleTartC> a <http://bakery.com/ns#NonGlutenFreeBakedGood>, <http://bakery.com/ns#VeganBakedGood> . merged into the data graph or as a inference graph.

    Is this feature available?

    opened by JoelBender 15
  • Strings with language tags errantly trigger validation errors

    Strings with language tags errantly trigger validation errors

    If a PropertyShape is declared with sh:datetype xsd:string, instance data with a string with a language tag will trigger a validation error, declaring the string to not be of type xsd:string. This appears to be isolated to sh:datatype constraints; sh:nodeKind does not appear to influence this behavior.

    I will be filing a PR in a moment to add a demonstration test, after getting an Issue number.

    opened by ajnelson-nist 14
  • Problem when validating xsd:float

    Problem when validating xsd:float

    Hi there,

    Validating an xsd:float gives me an unexpected validation report. I am using "PySHACL Version: 0.19.0".

    Example:

    shapes graph "shapes.json":

    {
      "@context": {
        "owl": "http://www.w3.org/2002/07/owl#",
        "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
        "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "skos": "http://www.w3.org/2004/02/skos/core#",
        "prov": "http://www.w3.org/ns/prov#",
        "dcat": "http://www.w3.org/ns/dcat#",
        "sh": "http://www.w3.org/ns/shacl#",
        "shsh": "http://www.w3.org/ns/shacl-shacl#",
        "dcterms": "http://purl.org/dc/terms/",
        "schema": "http://schema.org/",
        "rescs": "http://rescs.org/"
      },
      "@graph": [
        {
          "@id": "rescs:dash/monetaryamount/MonetaryAmountShape",
          "@type": "sh:NodeShape",
          "rdfs:comment": {
            "@type": "xsd:string",
            "@value": "A monetary value or range. This type can be used to describe an amount of money such as $50 USD, or a range as in describing a bank account being suitable for a balance between £1,000 and £1,000,000 GBP, or the value of a salary, etc. It is recommended to use [[PriceSpecification]] Types to describe the price of an Offer, Invoice, etc."
          },
          "rdfs:label": {
            "@type": "xsd:string",
            "@value": "Monetary amount"
          },
          "sh:property": {
            "sh:datatype": {
              "@id": "xsd:float"
            },
            "sh:description": "The value of the quantitative value or property value node.\\\\n\\\\n* For [[QuantitativeValue]] and [[MonetaryAmount]], the recommended type for values is 'Number'.\\\\n* For [[PropertyValue]], it can be 'Text;', 'Number', 'Boolean', or 'StructuredValue'.\\\\n* Use values from 0123456789 (Unicode 'DIGIT ZERO' (U+0030) to 'DIGIT NINE' (U+0039)) rather than superficially similiar Unicode symbols.\\\\n* Use '.' (Unicode 'FULL STOP' (U+002E)) rather than ',' to indicate a decimal point. Avoid using these symbols as a readability separator.",
            "sh:maxCount": {
              "@type": "xsd:integer",
              "@value": 1
            },
            "sh:minCount": {
              "@type": "xsd:integer",
              "@value": 1
            },
            "sh:minExclusive": 0,
            "sh:name": "value",
            "sh:path": {
              "@id": "schema:value"
            }
          },
          "sh:targetClass": {
            "@id": "schema:MonetaryAmount"
          }
        }
      ]
    }
    

    data sample "monetaryamount.json":

    {
      "@context": {
        "@vocab": "http://schema.org/",
        "xsd": "http://www.w3.org/2001/XMLSchema#"
      },
      "@type": "MonetaryAmount",
      "value": {
        "@type": "xsd:float",
        "@value": 100000
      }
    }
    

    pyshacl -sf json-ld -s shapes.json -df json-ld monetaryamount.json gives me:

    Validation Report Conforms: False Results (1): Constraint Violation in DatatypeConstraintComponent (http://www.w3.org/ns/shacl#DatatypeConstraintComponent): Severity: sh:Violation Source Shape: [ sh:datatype xsd:float ; sh:description Literal("The value of the quantitative value or property value node.\n\n* For [[QuantitativeValue]] and [[MonetaryAmount]], the recommended type for values is 'Number'.\n* For [[PropertyValue]], it can be 'Text;', 'Number', 'Boolean', or 'StructuredValue'.\n* Use values from 0123456789 (Unicode 'DIGIT ZERO' (U+0030) to 'DIGIT NINE' (U+0039)) rather than superficially similiar Unicode symbols.\n* Use '.' (Unicode 'FULL STOP' (U+002E)) rather than ',' to indicate a decimal point. Avoid using these symbols as a readability separator.") ; sh:maxCount Literal("1", datatype=xsd:integer) ; sh:minCount Literal("1", datatype=xsd:integer) ; sh:minExclusive Literal("0", datatype=xsd:integer) ; sh:name Literal("value") ; sh:path schema1:value ] Focus Node: [ :value Literal("100000", datatype=xsd:float) ; rdf:type :MonetaryAmount ] Value Node: Literal("100000", datatype=xsd:float) Result Path: schema1:value Message: Value is not Literal with datatype xsd:float

    Changing the @value to 100000.0 or "100000" makes it pass. However, I think all three variants should be valid, no?

    I tried the example above on https://shacl.org/playground/ which worked fine.

    Could you tell me whether I am doing something wrong or this is a bug?

    Thanks a lot!

    opened by tobiasschweizer 13
  • official pySHACL Docker image?

    official pySHACL Docker image?

    Is there an official pySHACL image and if not, could you release one and upload it for example at DockerHub? I could release one myself if you prefer but I think there are some advantages if the image is created and released by the developers of the project.

    opened by KonradHoeffner 11
  • owl:imports Content Negotiation

    owl:imports Content Negotiation

    Hi there

    I've been using pyshacl since a few days and I like it!

    I noticed the --imports flag and wanted to try it right away after having read about it in #18.

    So I tried the following:

    @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
    @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
    @prefix schema: <http://schema.org/> .
    @prefix sh: <http://www.w3.org/ns/shacl#> .
    @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
    @prefix owl: <http://www.w3.org/2002/07/owl#> .
    
    <http://datashapes.org/sh/tests/core/complex/personexample.test>
      rdf:type owl:Ontology ;
      rdfs:label "Test of personexample" ;
      owl:imports <http://datashapes.org/schema> ;
    .
    
    schema:PersonShape
        a sh:NodeShape ;
        sh:targetClass schema:Person ;
        sh:property [
            sh:path schema:gender ;
            sh:in ( "female" "male" ) ;
        ] .
    

    Which gives me the error:

    Bad syntax (expected '.' or '}' or ']' at end of statement) at ^ in: "b'\n\n \n '^b'<META http-equiv="Content-Type" content="text/html; charset='..." File "/usr/local/lib/python3.9/site-packages/rdflib/graph.py", line 1256, in parse raise ParserError( rdflib.exceptions.ParserError: Could not guess RDF format for <rdflib.parser.InputSource object at 0x10f9f1730> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.

    I figured that the source of the problem is that it got HTML instead of Turtle.

    So I tried:

    ...
    owl:imports <http://datashapes.org/schema.ttl> ; --> note the ttl extension
    ...
    

    And got:

    rdflib.plugins.parsers.notation3.BadSyntax: at line 4 of <>: Bad syntax (expected '.' or '}' or ']' at end of statement) at ^ in: "...b'ttp://topbraid.org/tosh">\n <owl:Ontology rdf:about="">\n '^b'sh:declare\n sh:PrefixDeclaration\n <sh:prefix'..." raise ParserError( rdflib.exceptions.ParserError: Could not guess RDF format for <rdflib.parser.InputSource object at 0x110035cd0> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.

    This time it was XML instead of Turtle (https://www.topbraid.org/tosh). http://datashapes.org/schema.ttl imports http://datashapes.org/dash which imports https://www.topbraid.org/tosh. Manually, I can easily figure out that it should be https://www.topbraid.org/tosh.ttl but I have no control over this using pyshacl.

    So my question is: Is there some mechanism for content negotiation (HTTP accept header) or some convention regarding the file extension pyshacl could use when processing owl:imports to avoid the problem described above?

    opened by tobiasschweizer 10
  • Remove (soft)dependency on rdflib_jsonld

    Remove (soft)dependency on rdflib_jsonld

    https://github.com/RDFLib/pySHACL/issues/92 fixed RDFLib 6.0, but a big change in RDFlib 6.0 was integrating rdflib_jsonld into RDFlib itself: https://github.com/RDFLib/rdflib/pull/1354

    Due to this, the rdflib_jsonld project has been archived and is not maintained anymore: https://github.com/RDFLib/rdflib-jsonld#archived

    Since rdflib_jsonld cannot be installed anymore with the newest version of setuptools, since it has use_2to3 set to true, see https://github.com/pypa/setuptools/issues/2769, this makes pySHACL unusable with JSON-LD with the newest version of setuptools.

    Graph().serialize(format='json-ld') should just work out of the box now with RDFLib 6.0.

    One slightly related issue (but irrelevant if the move away from rdflib_jsonld is done) is that the code in https://github.com/RDFLib/pySHACL/blob/master/pyshacl/rdfutil/load.py#L21-L26 doesn't work:

    File "/home/zenon/DEV/ETH/renku-python/renku/core/utils/shacl.py", line 21, in <module>
        from pyshacl import validate
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/__init__.py", line 4, in <module>
        from .shapes_graph import ShapesGraph
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/shapes_graph.py", line 7, in <module>
        from .constraints.constraint_component import CustomConstraintComponentFactory
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/constraints/__init__.py", line 4, in <module>
        from pyshacl.constraints.constraint_component import ConstraintComponent
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/constraints/constraint_component.py", line 36, in <module>
        from pyshacl.rdfutil import stringify_node
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/rdfutil/__init__.py", line 7, in <module>
        from .load import get_rdf_from_web, load_from_source  # noqa: F401
      File "/home/zenon/.pyenv/versions/renku-python/lib/python3.7/site-packages/pyshacl/rdfutil/load.py", line 22, in <module>
        import rdflib_jsonld  # noqa: F401
    ModuleNotFoundError: No module named 'rdflib_jsonld'
    

    It raises a ModuleNotFoundError for me, not an IndexError on Python 3.7.4

    opened by Panaetius 10
  • sh:not's severity defaults to sh:violation

    sh:not's severity defaults to sh:violation

    Unless I have overlooked something, the SHACL spec does not state that the severity of a sh:not (a property shape) is by default sh:violation. Therefore, I would think that a sh:not with sh:severity sh:Warning should return warnings. That is not the case when testing with this minimal example below:

    @prefix sh: <http://www.w3.org/ns/shacl#> .
    @prefix ex: <http://example.org/ex#> .
    
    ex:Foo ex:prop1 "a" .
    
    ex:FooShape
        sh:targetNode ex:Foo ;
        sh:not [
            a sh:PropertyShape ;
            sh:path ex:prop1 ;
            sh:hasValue "a" ;
            sh:severity sh:Warning ;
        ] ;
    .
    

    The output is:

    $ pyshacl -s foo.ttl foo.ttl
    Validation Report
    Conforms: False
    Results (1):
    Constraint Violation in NotConstraintComponent (http://www.w3.org/ns/shacl#NotConstraintComponent):
            Severity: sh:Violation
            Source Shape: ex:FooShape
            Focus Node: ex:Foo
            Value Node: ex:Foo
            Message: Node ex:Foo conforms to shape [ rdf:type sh:PropertyShape ; sh:hasValue Literal("a") ; sh:path ex:prop1 ; sh:severity sh:Warning ]
    
    opened by chrdebru 10
  • Resource of http://www.w3.org/ns/shacl#value is empty in validation report

    Resource of http://www.w3.org/ns/shacl#value is empty in validation report

    In the report the resource found in a http://www.w3.org/ns/shacl#value is empty, see the json below.

    [
      {
        ...
        "@type": [
          "http://www.w3.org/ns/shacl#ValidationResult"
        ],
        "http://www.w3.org/ns/shacl#focusNode": [
          {
            "@id": "http://vangoghmuseum.nl/data/artwork/d0005V1962"
          }
        ],
        ...
        "http://www.w3.org/ns/shacl#value": [
          {
            "@id": "_:N6087b61f1f1d44e08519420c185ba3f2"
          }
        ]
      },
      {
        "@id": "_:N6087b61f1f1d44e08519420c185ba3f2"
      },
    

    This report is the result of a propertyShape with a sh:node constraint. The first validation result in the example contains the information of the shape containing the sh:node. This fine. The value (N6087b61f1f1d44e08519420c185ba3f2) should contain the information of the result for the sh:node. I confirmed this in TopBraid.

    opened by michielhildebrand 10
  • Command-line use does not work in Windows

    Command-line use does not work in Windows

    The path is not getting interpreted correctly: file://c:\my\full\path\test.ttl/ does not look like a valid URI, trying to serialize this will break.

    I've tried with forward slashes, backslashes, no slashes (all files in current directory), full filespec (with and without c:), etc. Couldn't get any of them to work.

    Windows 10.

    opened by nextcen-dgemoets 10
  • SHACL and subPropertyOf

    SHACL and subPropertyOf

    Hi @ashleysommer ,

    I am trying to write a small "Gluing" SHACL shapes graph that combines two ontologies that have their own SHACL shapes. One of the gluing points is I want to make a ClassAB that is a subclass of classA and classB. classA has a certain property, propertyA, with a minimum-cardinality constraint, sh:min 1, and classB has an analagous property, propertyB, without that cardinality constraint. My use case happens to call for, whenever propertyA is assigned, propertyB should also be assigned with the same target. (The benefits to doing this lie in the class hierarchy above and below this "Gluing point.")

    This feels to me like a good opportunity to try some multiple-inheritance with classes and with properties. However, when I tried defining a subproperty propertyAB, this was not being recognized by pyshacl as a usage of propertyA or propertyB.

    I think I know the answer is basically "Yes, that's right," due to this quote from @HolgerKnublauch:

    ... sh:paths do not look at rdfs:subPropertyOf ...

    and also from not seeing the word "subProperty" appear anywhere in the SHACL specification.

    I can resort to a some scripting to save a little bit of typing, and/or sh:sparql to still make the data guarantees I want to make. But, is it true that rdfs:subPropertyOf will just be ignored by SHACL?


    This issue can be reproduced with the following shapes graph and instance data. The expected sh:focusNodes are the ones with the comment of XFAIL. The ones actually returned are:

    • kb:classApropertyAB
    • kb:classApropertyB
    • kb:classBpropertyAB
    • kb:classBpropertyA

    shapes.ttl:

    @prefix ex: <http://example.org/ontology/> .
    @prefix owl: <http://www.w3.org/2002/07/owl#> .
    @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
    @prefix sh: <http://www.w3.org/ns/shacl#> .
    
    ex:ClassA
    	a
    		owl:Class ,
    		sh:NodeShape 
    		;
    	sh:property [
    		a sh:PropertyShape ;
    		sh:minCount 1 ;
    		sh:nodeKind sh:BlankNodeOrIRI ;
    		sh:path ex:propertyA
    	] ;
    	sh:targetClass ex:ClassA ;
    	.
    
    ex:ClassB
    	a
    		owl:Class ,
    		sh:NodeShape 
    		;
    	sh:property [
    		a sh:PropertyShape ;
    		sh:minCount 1 ;
    		sh:nodeKind sh:BlankNodeOrIRI ;
    		sh:path ex:propertyB
    	] ;
    	sh:targetClass ex:ClassB ;
    	.
    
    ex:propertyA
    	a owl:ObjectProperty ;
    	.
    
    ex:propertyB
    	a owl:ObjectProperty ;
    	.
    
    ex:propertyAB
    	a owl:ObjectProperty ;
    	rdfs:subPropertyOf
    		ex:propertyA ,
    		ex:propertyB
    		;
    	.
    

    data.ttl:

    @prefix ex: <http://example.org/ontology/> .
    @prefix kb: <http://example.org/kb/> .
    @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
    
    kb:classApropertyA
    	a ex:ClassA ;
    	rdfs:comment "PASS" ;
    	ex:propertyA [] ;
    	.
    
    kb:classApropertyAB
    	a ex:ClassA ;
    	rdfs:comment "PASS" ;
    	ex:propertyAB [] ;
    	.
    
    kb:classApropertyB
    	a ex:ClassA ;
    	rdfs:comment "XFAIL" ;
    	ex:propertyB [] ;
    	.
    
    kb:classBpropertyA
    	a ex:ClassB ;
    	rdfs:comment "XFAIL" ;
    	ex:propertyA [] ;
    	.
    
    kb:classBpropertyAB
    	a ex:ClassB ;
    	rdfs:comment "PASS" ;
    	ex:propertyAB [] ;
    	.
    
    kb:classBpropertyB
    	a ex:ClassB ;
    	rdfs:comment "PASS" ;
    	ex:propertyB [] ;
    	.
    

    The command I used to see the failure was:

    pyshacl -s shapes.ttl data.ttl
    
    opened by ajnelson-nist 9
  • Add tests to confirm detection of various integer types

    Add tests to confirm detection of various integer types

    This patch series will add tests to confirm behaviors of XSD integer types aside from xsd:integer. It stems from trying to test a sample with a literal "0"^^xsd:positiveInteger and not receiving an error.

    It is not entirely clear to me if corrections from this patch series should be in pySHACL or upstream in RDFLib.

    opened by ajnelson-nist 9
  • Trimming SHACL from the ontology graph

    Trimming SHACL from the ontology graph

    This is a port of a comment I'd made on Sep. 16 on Issue 89. I've migrated the comment here to avoid expanding the scope of that ticket.

    I've come across undesired behavior related to pySHACL's ontology graph and shapes graph mix-in strategy. The short of it is, my community adopted a SHACL-based mechanism to review some OWL requirements and cross-check SHACL and OWL definitions with general patterns. (The shapes graph for this is here. We'd also appreciate hearing if you've seen something like this elsewhere; we couldn't find something like it.) My community's ontology is managed as Turtle files, one per namespace, and each Turtle file includes all OWL and SHACL statements for concepts in that namespace.

    What we didn't appreciate is that this incurs a significant cost on reviewing all data graphs, due to the mix-in strategy, so our "TBox"-targeting tests also run on "ABox" data review because pyshacl mixes the "ontology" TBox and ABox stuff together. With our ontology (a few hundred classes and properties, each with associated SHACL shapes), this is a 20--30 second cost per pyshacl run, guaranteed to be redundantly re-reviewing the ontology.

    So, I'd like to discuss whether the whole ontology mix-in is still the right approach in all cases.

    The case my community is facing at the moment is our ontology is written including SHACL shapes and OWL definitions in the same graph file, so we feed SHACL statements into .validate()'s ont_graph keyword argument. Question: Would the ontology graph pySHACL uses (ont_graph) ever be expected to include SHACL statements? My guess is no.

    I was considering whether there could be a further reasonable trim based on the user's inferencing capabilities request, but I think the answer there may be no. I suspect SHACL-SPARQL makes the whole ontology graph always potentially necessary, though I can't think of a good example at the moment - maybe linking via some annotation property only available in the ontology, or searching for deprecated classes?

    For now, my one question is just the SHACL-in-ontology-graph question: Can SHACL be safely trimmed from the ontology graph?

    opened by ajnelson-nist 1
  • Influence of meta_shacl

    Influence of meta_shacl

    Dear RDFlib people!

    I have a combination of SHACL shapes. With "meta_shacl= True" they are processed but leaving a trace in the wake.

    With "meta_shacl= False" I get only a stack trace.

    Evidence below.

    I do not understand this behavior. Can anyone please shed light. I would have expect the behavior reversed.

    Cheers, Volker

    Setting meta_shacl= True :

    /home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/bin/python /home/volker/workspace/PYTHON5/SHACL_validation/shacl/validate.py 
    Shacl File does not validate against the Shacl Shapes Shacl file.
    Validation Report
    Conforms: False
    Results (21):
    Constraint Violation in MaxCountConstraintComponent (http://www.w3.org/ns/shacl#MaxCountConstraintComponent):
    	Severity: sh:Violation
    	Source Shape: [ sh:maxCount Literal("0", datatype=xsd:integer) ; sh:path sh:path ]
    	Focus Node: [ sh:hasValue <http://publications.europa.eu/resource/dataset/planned-availability> ; sh:path skos:inScheme ]
    	Result Path: sh:path
    	Message: More than 0 values on [ sh:hasValue <http://publications.europa.eu/resource/dataset/planned-availability> ; sh:path skos:inScheme ]->sh:path
    Constraint Violation in MaxCountConstraintComponent (http://www.w3.org/ns/shacl#MaxCountConstraintComponent):
    	Severity: sh:Violation
    	Source Shape: [ sh:maxCount Literal("0", datatype=xsd:integer) ; sh:path sh:path ]
    	Focus Node: [ sh:hasValue <http://dcat-ap.de/def/licenses> ; sh:path skos:inScheme ]
    	Result Path: sh:path
    	Message: More than 0 values on [ sh:hasValue <http://dcat-ap.de/def/licenses> ; sh:path skos:inScheme ]->sh:path
    Constraint Violation in MaxCountConstraintComponent (http://www.w3.org/ns/shacl#MaxCountConstraintComponent):
    	Severity: sh:Violation
    	Source Shape: [ sh:maxCount Literal("0", datatype=xsd:integer) ; sh:path sh:path ]
    	Focus Node: [ sh:hasValue <http://publications.europa.eu/resource/dataset/planned-availability> ; sh:path skos:inScheme ]
    	Result Path: sh:path
    	Message: More than 0 values on [ sh:hasValue <http://publications.europa.eu/resource/dataset/planned-availability> ; sh:path skos:inScheme ]->sh:path
    

    then a trace.

    Setting meta_shacl= False :

    /home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/bin/python /home/volker/workspace/PYTHON5/SHACL_validation/shacl/validate.py 
    Traceback (most recent call last):
      File "/home/volker/workspace/PYTHON5/SHACL_validation/shacl/validate.py", line 31, in <module>
        conforms, report_graph, report_text = pyshacl.validate(
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/validate.py", line 435, in validate
        conforms, report_graph, report_text = validator.run()
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/validate.py", line 265, in run
        _is_conform, _reports = s.validate(
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/shape.py", line 511, in validate
        _is_conform, _reports = c.evaluate(target_graph, focus_value_nodes, _e_p)
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 108, in evaluate
        _nc, _r = _evaluate_property_shape(p_shape)
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 102, in _evaluate_property_shape
        _is_conform, _r = prop_shape.validate(target_graph, focus=v, _evaluation_path=_evaluation_path[:])
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/shape.py", line 511, in validate
        _is_conform, _reports = c.evaluate(target_graph, focus_value_nodes, _e_p)
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 196, in evaluate
        _nc, _r = _evaluate_node_shape(n_shape)
      File "/home/volker/workspace/venvs/shacl-v-H5Zt4z-py3.9/lib/python3.9/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 182, in _evaluate_node_shape
        raise ReportableRuntimeError(
    pyshacl.errors.ReportableRuntimeError: Shape pointed to by sh:node does not exist or is not a well-formed SHACL NodeShape.
    
    Process finished with exit code 1
    
    opened by volkerjaenisch 2
  • Fails on xsd:dateTimeStamp rdfs:subClassOf xsd:dateTime. Howto enforce inheritance?

    Fails on xsd:dateTimeStamp rdfs:subClassOf xsd:dateTime. Howto enforce inheritance?

    Dear RDFlib developers!

    At first thank you so much! Without your code our Open-Data Portal https://datenadler.de would never have taken flight. We do not use CKAN, we programmed from scratch. Currently we are the second most populated Open-Data portal in Germany.

    But now we need SHACL. pySHACL generally runs fine but I seems that we are running into some detail problems.

    This may be an exotic case. It does not work on other SHACL processors, too. So it may be a limitation of SHACL itself.

    So this is not a bug report but more stating of a SHACL fact.

    Given the following data

    xsd:dateTimeStamp rdfs:subClassOf xsd:dateTime .
    
    <https://geobasis-bb.de#dcat_Dataset_568978c5-fa73-48d1-a6f9-487aabdc1aef> a dcat:Dataset;
      dct:description "Für die Digitalen Topographischen Karten werden Vektordaten.."@de;
      dct:identifier "568978c5-fa73-48d1-a6f9-487aabdc1aef";
      dct:modified "2022-11-17T09:37:25.626789"^^xsd:dateTimeStamp;
    
    

    and the following shape

    :DateOrDateTimeDataType_Shape
        a sh:NodeShape ;
        rdfs:comment "Date time date disjunction shape checks that a datatype property receives a temporal value: date, dateTime, gYear or gYearMonth literal" ;
        rdfs:label "Date time date disjunction" ;
        sh:message "The values must be data typed as either xsd:date, xsd:dateTime, xsd:gYear or xsd:gYearMonth" ;
        sh:or ([
                sh:datatype xsd:date
            ]
            [
                sh:datatype xsd:dateTime
            ]
    		[
                sh:datatype xsd:gYear
            ]
    		[
                sh:datatype xsd:gYearMonth
            ]
        ) .
    

    OK, the files are a bit more complex but it boils down to that.

    We get an 
    
    Validation Report
    Conforms: False
    Results (20):
    Constraint Violation in NodeConstraintComponent (http://www.w3.org/ns/shacl#NodeConstraintComponent):
    	Severity: sh:Violation
    	Source Shape: :Dataset_Property_dct_issued
    	Focus Node: <https://geobasis-bb.de#dcat_Dataset_568978c5-fa73-48d1-a6f9-487aabdc1aef>
    	Value Node: Literal("2022-11-17T09:37:25.626872" = None, datatype=xsd:dateTimeStamp)
    	Result Path: dct:issued
    	Message: Value does not conform to Shape :DateOrDateTimeDataType_Shape
    

    Is it possible with pySHACL to inject the inheritance information:

    xsd:dateTimeStamp rdfs:subClassOf xsd:dateTime .

    If so, how? Any help appreciated.

    Cheers, Volker

    opened by volkerjaenisch 4
  • pySHACL cannot read resources from chunked HTTPResponses

    pySHACL cannot read resources from chunked HTTPResponses

    Dear PySHACL Developers!

    PySHACL fails to open a http resource if it is chunked. I propose a quickfix and additional info below the stacktrace.

    Code to reproduce:

    from pyshacl.rdfutil import load_from_source
    
    load_from_source('http://publications.europa.eu/resource/dataset/planned-availability')
    

    Stacktrace:

    Traceback (most recent call last):
      File "/usr/lib/python3.9/xml/sax/expatreader.py", line 217, in feed
        self._parser.Parse(data, isFinal)
    xml.parsers.expat.ExpatError: syntax error: line 1, column 0
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/volker/workspace/PYTHON2/shacl/sandbox/error2.py", line 3, in <module>
        load_from_source('http://publications.europa.eu/resource/dataset/planned-availability')
      File "/home/volker/workspace/venvs/shacl-WIAd6my0-py3.9/lib/python3.9/site-packages/pyshacl/rdfutil/load.py", line 352, in load_from_source
        target_g.parse(source=cast(IO[bytes], _source), format=rdf_format, publicID=public_id)
      File "/home/volker/workspace/venvs/shacl-WIAd6my0-py3.9/lib/python3.9/site-packages/rdflib/graph.py", line 1330, in parse
        parser.parse(source, self, **args)
      File "/home/volker/workspace/venvs/shacl-WIAd6my0-py3.9/lib/python3.9/site-packages/rdflib/plugins/parsers/rdfxml.py", line 604, in parse
        self._parser.parse(source)
      File "/usr/lib/python3.9/xml/sax/expatreader.py", line 111, in parse
        xmlreader.IncrementalParser.parse(self, source)
      File "/usr/lib/python3.9/xml/sax/xmlreader.py", line 125, in parse
        self.feed(buffer)
      File "/usr/lib/python3.9/xml/sax/expatreader.py", line 221, in feed
        self._err_handler.fatalError(exc)
      File "/usr/lib/python3.9/xml/sax/handler.py", line 38, in fatalError
        raise exception
    xml.sax._exceptions.SAXParseException: <unknown>:1:0: syntax error
    
    Process finished with exit code 1
    
    

    The case of the problem is in load.py : 161

                    filename = resp.geturl()
                    fp = resp.fp  # type: BufferedIOBase
                    source_was_open = False
                    source = open_source = fp
    

    Here the filepointer resp.fp of the response is used as source for the parser. This goes well as long as the response is not chunked.

    If the response is chunked there is a difference between

    resp.read(20) b'<rdf:RDF\n xmlns:r'

    and

    resp.fp.read(20) b'3ae7\r\n<rdf:RDF\n x'

    This is due to the fact that fp is the low level filepointer, which has to be used differently with chunking. b'3ae7\r \n' is position of the next chunk.

    Patching load.py : 161 to

    fp = resp solves the problem for this case.

    But this code is complex and maybe some other use cases do need the use of the filepointer.

    Cheers,

    Volker

    opened by volkerjaenisch 1
  • Integrate existing reusable GitHub Actions workflow?

    Integrate existing reusable GitHub Actions workflow?

    I would like to share a reusable GitHub Actions workflow to use pySHACL in a CI pipeline. The repository is https://github.com/KonradHoeffner/shacl and it is on the market place as konradhoeffner/shacl@v1. I found it really useful for our projects and would like to know if this is useful for the pySHACL developers / userbase. If yes, are you interested in integrating it into pySHACL and/or the RDFLib organization? If you are interested and it makes sense to integrate it directly into this repository, then I could create a pull request. However I'm not sure how that would be managed in the best way, because right now it installs the newest version <2 from pip, and maybe the action number should be synchronized with the pySHACL version.

    Example

    On https://github.com/hitontology/ontology you can see an example of how it works.

    Workflow

    The workflow .github/workflows/shacl.yaml calls the reusable workflow konradhoeffner/shacl@master (could be @v1 as well).

    name: SHACL
    
    on:
      workflow_dispatch:
      push:
        branches:
          - dist
    
    jobs:
      shacl:
        runs-on: ubuntu-latest
        steps:
          - name: Checkout code
            uses: actions/checkout@v3
            with:
              ref: 'dist'
    
          - name: Build and Validate
            uses: konradhoeffner/shacl@master
            with:
              data: all.ttl
              shacl: shacl.ttl
    

    Status report on success

    Screenshot from 2022-09-21 10-08-21

    Status report on failure from another repository

    Screenshot from 2022-09-21 10-09-50

    Badge

    shaclbadge

    Technical details and history

    At first there was a separate workflow that built a Dockerfile that just installed pySHACL on top of a Python image and included a custom entry point script that calls pySHACL in a loop over the potentially multiple data files and that creates the right output strings to interface with GitHub to get nice output like errors, warnings and info notifications and groups. The reason to use a Dockerfile was that GitHub actions use a lot of time and clutter the log with messages to install dependencies and I wanted a quick build time and a clean log with only the statements that are interesting for the user who is validating SHACL data and is not interested in logging about installing things. However I learned that while Docker is amazing for many things, it is not the preferred GitHub-way of doing things, because an action already has a "runner" which is kind of like a virtual machine / container, and using Docker on top of that does seem to integrate that well. For example, the idea was to build the Docker image only once and save all the time for setup, but now GitHub actions would create the Docker image at every run, ruining all the potential benefits. So I had to create another workflow inside the repository that built the Dockerfile and deployed it to the GitHub container registry. While this was a few seconds quicker and the output more clean, this approach was much too convoluted and could get out of sync between the Docker image and the action itself, for example when running an older version of the workflow. So the current version v1 replaces the Dockerfile with a composite action that installs Python and then runs the entrypoint script directly. While that created a few problems, for example that a workflow referencing the shared workflow from another repository couldn't access the entrypoint script or that there was no file at the target repository to create a hash for the setup-python cache for, this was solvable by disabling the setup-python cache and using actions/cache instead and by using the ${{github.action_path}}.

    opened by KonradHoeffner 0
Releases(v0.20.0)
  • v0.20.0(Sep 8, 2022)

    Note, while this is a normal 0.x release, it also acts as the v1.0 release candidate.

    That means, if no glaring bugs or issues are found in this release after two weeks, this version will be re-released as PySHACL v1.0.

    In this release:

    Fixed

    • Ill-typed/Ill-formed literals now fail the DataType test, as expected
      • Requires RDFLib v6.2.0+
      • Fixes #140 (and possibly fixes #151)
      • Unskipped one of the remaining skipped shacl-test-suite SHT tests (datatype-ill-formed_ttl.ttl)
    • Fixed detection of recursion to be more lenient of deliberately recursive (but not infinitely recursive) shapes.
      • Fixes #154
    • MetaShacl works again now with RDFLib >= v6.2.0
      • Fixes #153
    • Fixed typing issues affecting interoperability of new version of RDFLib and PySHACL.

    Changed

    • RDFLib v6.2.0 or greater is now required to run PySHACL
      • This new version of RDFLib implements the ill-typed Literals feature, that helps with sh:datatype constraint validation.
      • Removing support for older versions of RDFLib allows PySHACL to implement new features, and have less unnecessary code
    • Bumped to using new Poetry v1.2.0 (or newest poetry-core v1.1.0)
      • Changed pytest-cov and coverage tests to be optional dependencies for dev
    • Bumped version of Black to 22.8.0, and re-blacked all files
    • Removed old monkey patches, no longer needed for the latest version of RDFLib
    • Removed bundled "Memory2" store, now using the default "Memory" from RDFLib
      • Regenerated bundled pickled triplestores, to use Memory instead of Memory2
    • Updated official dockerfile with newest version of PySHACL and RDFLib
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.20.0-py3-none-any.whl(1.14 MB)
    pyshacl-0.20.0.tar.gz(1.19 MB)
  • v0.19.1(Jun 30, 2022)

    Note, while this is a normal 0.x release, it also acts as the v1.0 release candidate.

    That means, if no glaring bugs or issues are found in this release after two weeks, this version will be re-released as PySHACL v1.0.

    Fixed

    • CLI Output Table formatting crashed when report graph did not contain a resultMessage
      • Fixes #145
    • Executing advanced-mode triples rules can sometimes skip the graph clone step, and incorrectly emits new triples directly into the input data-graph
      • Discovered when investigating #148

    Changed

    • Executing advanced triples rules no longer incorrectly emits new triples directly into the input data-graph
      • This may been seen as a breaking change, if your workflow relied on this incorrect behaviour.
      • If you really the rules engine to emit new triples into your input data graph, use the inplace validator option.
    • Updated built-in schema.ttl file to newer version that doesn't have UTF-8 encoding issues

    Added

    • Official Dockerfile is now included in the repository
      • Thanks @KonradHoeffner; Fixes #135
      • Published to dockerhub at ashleysommer/pyshacl
      • docker pull docker.io/ashleysommer/pyshacl:latest
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.19.1-py3-none-any.whl(1.15 MB)
    pyshacl-0.19.1.tar.gz(1.19 MB)
  • v0.19.0(Mar 12, 2022)

    Note, while this is a normal 0.x release, it also acts as the v1.0 release candidate.

    That means, if no glaring bugs or issues are found in this release after two weeks, this version will be re-released as PySHACL v1.0.

    In this release:

    Fixed

    • Fixed a long-standing oversight where ShapeLoadErrors and ConstraintLoadErrors were not reported correctly when running PySHACL in CLI mode.
      • Sorry about that. Thanks lots of people for reporting this over the last year. I wish I fixed it sooner.
    • Fixed a long-standing bug where using $PATH in a sh:sparql query on a PropertyShape would not work correctly.
      • Fixes #124, Thanks @Martijn-Y-ai
    • Fixed a long-standing bug, that allows PySHACL to more reliably determine if graph source is a file path, or a graph string.
      • Fixes #132, Thanks @Zezombye
    • Fixed an issue where sh:pattern could not be applied to a Literal that was not an xsd:string or URI.
      • Fixes #133, Thanks @nicholascar
    • Fixed the outdated/incorrect reported when a PropertyShape's sh:path value gets an unknown path type.
      • Fixes #129, Thanks @edmondchuc

    Added

    • New --allow-infos option in CLI mode and Python Module mode.
      • This is like --allow-warnings except it only allows violations with severity of sh:Info.
      • (--allow-warnings continues to allow both sh:Warning and sh:Info as it used to.)
      • Fixes #126, Thanks @ajnelson-nist
    • SPARQL-based Constraints can now substitute arbitrary bound SPARQL variables into their sh:message
      • Fixes #120

    Changed

    • --allow-infos and --allow-warnings can now also be enabled with --allow-info and --allow-warning respectively.
    • Removed Snyk check on CI/CD pipeline, because there is an RDFLib issue blocking Snyk on PySHACL from passing.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.19.0-py3-none-any.whl(1.15 MB)
    pyshacl-0.19.0.tar.gz(1.19 MB)
  • v0.18.1(Jan 25, 2022)

  • v0.18.0(Jan 13, 2022)

    Added

    • Added Python 3.10 support (when using RDFLib v6.1.1 or greater)
    • Added more type hinting, to conform to the new type hinting added by RDFLib 6.1.1
    • Added Python 3.10 to test suite

    Changed

    • Subtle correction in the way sh:prefixs works with sh:declare on the given named ontology.
    • Bumped some min versions of libraries, to gain compatibility with Python 3.10

    Fixed

    • Fixed test for issue #76
    • Fixed #76 again (after fixed test)
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.18.0-py3-none-any.whl(1.14 MB)
    pyshacl-0.18.0.tar.gz(1.19 MB)
  • v0.17.3(Dec 13, 2021)

  • v0.17.2(Oct 25, 2021)

  • v0.17.1(Oct 11, 2021)

    Fixes

    • Handle transitive subclasses when evaluating sh:targetClass - @gtfierro
      • Fixes #96
    • Improve detection of RDF/XML files when loading unknown content
      • Fixes #98
    • Imported type stubs and resolved ALL MyPy issues! (this was a big effort)
    • Logic fixes in the dataset loader (thanks to inconsistencies exposed by MyPy)

    Changed

    • Add special cases to sh:dataclass constraint, when the given shape uses rdfs:Literal or rdfs:Dataclass as the dataclass to match on
      • Fixes #71

    Added

    • Add datashapes.org/schema as a built-in graph
      • Fixes #98
    • Added ability to pass a TextIO or TextIOWrapper object into the dataset loader
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.17.1-py3-none-any.whl(1.14 MB)
    pyshacl-0.17.1.tar.gz(1.18 MB)
  • v0.17.0(Sep 13, 2021)

    Notice

    This version of PySHACL requires RDFLib 6.0.0. As a direct result of that, this version of PySHACL also requires Python v3.7.

    Changed

    • Upped RDFLib min version to 6.0.0 in order to get built-in json-ld
    • Upped OWL-RL to min version 5.2.3 in order to remove json-ld dependency
    • Made min python version v3.7
    • Change black config to use python 3.7 compat code
    • Re-black and isort all source files
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.17.0-py3-none-any.whl(193.01 KB)
    pyshacl-0.17.0.tar.gz(246.28 KB)
  • v0.16.2.post1(Sep 13, 2021)

  • v0.16.2(Sep 12, 2021)

    Notice

    This is the last version of PySHACL to support RDFLib 5.0.0, subsequent releases of PySHACL will depend on RDFLib v6.0.0. As a direct result of that, this is also the last version of PySHACL to support Python v3.6.

    Changed

    • Pinned JSON-ld dep to <6.0 to avoid the tombstone release (so not to force rdflib 6.0)
    • Updated minimum Black version to 21.8b0 to fix a black bug
    • Re-black and isort all source files

    Fixed

    • Fixed detection of import error when loading json-ld module in RDF loader
    • Fixed Black bug with new version of black
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.16.2-py3-none-any.whl(193.05 KB)
    pyshacl-0.16.2.tar.gz(246.45 KB)
  • v0.16.1.post1(Aug 27, 2021)

  • v0.16.1(Aug 19, 2021)

    Added

    • ExpressionConstraintComponent is implemented!
      • Use your previously defined SHACL Functions to express complex constraints
      • Added DASH-tests for ExpressionConstraintComponent
      • Added advanced tests for ExpressionConstraintComponent, SHACLRules, and SHACLFunctions.
    • New Advanced features example, showcasing ExpressionConstraint and others features

    Changed

    • Allow sh:message to be attached to an expression block, without breaking its functionality
    • A SHACL Function within a SHACL Expression now must be a list-valued property.
    • Refactored node-expression and path-expression methods to be common and reusable code
    • Re-black and isort all source files
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.16.1-py3-none-any.whl(192.84 KB)
    pyshacl-0.16.1.tar.gz(244.55 KB)
  • v0.16.0(Aug 19, 2021)

  • v0.15.0(Jul 20, 2021)

  • v0.14.5(Jul 7, 2021)

    Added

    • Allow-Warnings is finally available. (Closes #64)
      • Setting this option puts PySHACL into a non-standard operation mode, where Shapes marked with severity of sh:Warning or sh:Info will not cause result to be invalid.
      • Despite the name, it allows both sh:Info and sh:Warning.
      • Try it with allow_warnings=True on validate() or -w in the CLI tool.

    Fixed

    • Fixed Abort Mode. (Fixes #75)
      • This optional mode allows the validator to exit early, on the first time your data fails to validate against a Constraint.
      • Name changed from abort_on_error to abort_on_first
      • Try it out with abort_on_first=True on validate() or --abort in the CLI tool.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.14.5-py3-none-any.whl(190.30 KB)
    pyshacl-0.14.5.tar.gz(241.51 KB)
  • v0.14.4(May 26, 2021)

  • v0.14.3_z(May 12, 2021)

  • v0.14.3(Feb 22, 2021)

    • Relaxed the Max Evaluation Depth from 28 to 30, we were seeing some real-world cases where meta-shacl was failing on large Shapes Graphs at 28 levels deep.
    • sh:namespace values can now be xsd:anyURI or xsd:string or "literal string", but now cannot be .
    • sh:order can now support xsd:decimal values and xsd:integer values, and can be interchanged at will.
    Source code(tar.gz)
    Source code(zip)
  • v0.14.2(Feb 22, 2021)

    Added

    • Potential speedups when executing validation by lazy-loading large modules which may never be required in a normal validation run.

    Fixed

    • Black and Flake8 issues outstanding from 0.14.1 release.
    • Workaround a RDFLib bug trying to import requests when requests is not required to be installed.
      • This bug will still be observed if you use SPARQLConstraints, SPARQLFunction or JSFunction features, but it can be worked around by simply installing requests in your python environment.
    Source code(tar.gz)
    Source code(zip)
  • v0.14.1(Dec 23, 2020)

    Release 0.14.1 - 2020-12-23

    Added

    • Inplace Mode, for when cloning your datagraph is undesirable
      • Normally pyshacl will create an in-memory copy of your datagraph before modifying it (when using ontology mixin, or inferencing features)
      • This might be unwanted if your datagraph is very large or remote and cloning it into memory is not a good option
      • Enabling inplace mode will bypass this clone step, and apply modification operations directly on your data_graph (use with caution!)
      • Enable with inplace=True kwarg on validate().
      • Inplace mode is not yet available via the CLI application, and perhaps doesn't even make sense to have it available there.

    Fixed

    • Inferencing will no longer incorrectly place expanded triples into your original data_graph, unless you enable 'inplace'
    • SHACL-JS loader will no longer fail if the regex module is not installed (it will fall back to using builtin re)
    • SHACL-Rule DASH-tests will now pass when the SHACL-rule is applied on multigraph (Dataset or ConjunctiveGraph)
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.14.1-py3-none-any.whl(187.79 KB)
    pyshacl-0.14.1.tar.gz(216.95 KB)
  • v0.14.0(Oct 14, 2020)

    Have you ever wanted to use Javascript to write SHACL constraints? Do you want to use Javascript to select custom SHACL targets? Have you the need to use Javascript to run SHACL Rules for emitting triples? Do you have the requirement to execute Javascript code from a SPARQL Function?

    If yes, then this is the release for you!

    Added

    • SHACL-JS Support!
    • Implements all of the features in the SHACL-JS SHACL Extension specification: https://www.w3.org/TR/shacl-js/
    • Includes:
      • JS Constraints
      • JS ConstraintComponents
      • JS SHACL Functions
      • JS SHACL Rules
      • JS Target
      • JS TargetType
    • To install it, make sure you do pip3 install pyshacl[js] to get the correct extra packages.

    Changed

    • Added JS flag to the CLI tool to enable SHACL-JS features
    • Updated README and FEATURES matrix

    Fixed

    • Fixes #43
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.14.0-py3-none-any.whl(186.95 KB)
    pyshacl-0.14.0.tar.gz(217.81 KB)
  • v0.13.3(Sep 11, 2020)

  • v0.13.2(Sep 10, 2020)

    Added

    • Added the ability for PySHACL to use baked in graphs instead of fetching them from a HTTP endpoint when a known graph is imported using owl:imports
      • This allows for time savings on graph-load and saves a HTTP request
      • Also allows us to embed fixed errata versions of files in place of release-time ones online

    Fixed

    • With new features, comes new bugs
    • With the ability to now load SPARQLFunctions, this removes the barrier for loading Schema.org SHACL in advanced mode
    • But when doing so revealed more issues. They are now fixed:
    • Fixed SPARQLConstraintComponent getting confused when shacl.ttl was loaded into your Shapes file using owl:imports
    • Fixed https://github.com/RDFLib/pySHACL/issues/61

    Changed

    • Refactored SPARQLConstraintComponent code, to allow for other custom constraint components in the future
      • This prevented SPARQLConstraintComponent getting confused when shacl.ttl was loaded into the Shapes file using owl:imports

    Note I know there are some mypi errors in this release. This does not affect runtime functionality. These typing errors will be fixed in the near future.

    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.13.2-py3-none-any.whl(168.44 KB)
    pyshacl-0.13.2.tar.gz(200.52 KB)
  • V0.13.1(Sep 7, 2020)

    Added

    • SPARQLTargetType
      • New SPARQL-based Target Type feature
      • The Paramaterisable form of SPARQLTarget from the SHACL Advanced Features spec
      • https://www.w3.org/TR/shacl-af/#SPARQLTargetType
    • Added a test for SPARQLTargetType - Theres none in the SHT suite, or the DASH suite.

    Changed

    • Refactored sh:parameter code in SPARQL-based Constraint Components, SHACLFunctions, and SPARQL-Based Target Types
      • They all now share a common SHACLParameter helper class, reducing code duplication
    • Refactored SPARQLQueryHelper
      • SPARQLQueryHelper internal class is now more helpful
      • query_helper can now extract param bindings into param-value pairs for parameterised queries
      • Reduces more code duplication
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.13.1-py3-none-any.whl(125.51 KB)
    pyshacl-0.13.1.tar.gz(158.75 KB)
  • v0.13.0(Sep 4, 2020)

    Added

    • New SHACL Advanced Specification Features!
    • All NodeExpressions can now be used in SHACL Rules
      • Focus Node (sh:this)
      • FilterShape (sh:filterShape)
      • Function Expressions (any sh:SHACLFunction and args)
      • Path Expressions (use sh:path in a NodeExpression)
      • Intersection Expressions (sh:intersection)
      • Union Expressions (sh:union)
    • SHACLFunctions (including SPARQLFunction)
      • Both SHACLFunction and SPARQLFunction are now fully implemented including unit tests and edge cases
      • SHACLFunctions are bound to PySHACL and can be used in SHACL Rules and NodeExpressions
      • SPARQLFunctions are bound to the RDFLib SPARQL Engine, so they can be used in other SPARQL queries
      • Read the manual for more info: https://www.w3.org/TR/shacl-af/#functions

    Fixed

    • Short versions of uris were sometimes not used in the Validation Report when they should've been
    • Checking results of some tests was being skipped! Lucky this wasn't letting through any SHACL errors.
    • Fixed error message when using sh:ignoredProperties on a node that isn't sh:closed issue #58
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.13.0-py3-none-any.whl(121.75 KB)
    pyshacl-0.13.0.tar.gz(154.95 KB)
  • v0.12.2(Aug 12, 2020)

    Fixed

    • In a validation report graph, when FocusNode and ValueNode are the same node, and are a blank node, when they get copied into the validation report graph they will have the same bnode id as each other.
    • Optimised the algorithm for copying different kinds of rdf nodes into the validation graph.

    Changed

    • When the FocusNode and ValueNode are copied into the validation graph from the data graph, they will try to keep the same bnode id they had before, if possible.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.12.2-py3-none-any.whl(115.76 KB)
    pyshacl-0.12.2.tar.gz(148.73 KB)
  • v0.12.1(Jul 22, 2020)

    Added

    • All SHACL Core constraints now have their own autogenerated sh:message.

      • This is used as a fallback when your Shape does not provide its own sh:message
      • See the new sh:resultMessage entries in the Validation Report output
      • These are hopefully more human-readable than the other fields of the Validation Report results
    • Added a copy of the implementation of the new 'Memory2' rdflib triplestore backend.

      • This when using Python 3.6 or above, this is faster than the default 'IOMemory' store by:
        • 10.3% when benchmarking validation with no inferencing
        • 17% when benchmarking validation with rdfs inferencing
        • 19.5% when benchmarking validation with rdfs+owlrl inferencing

    Changed

    • PySHACL is now categorised as Production/Stable.
      • This marks a level of maturity in PySHACL we are happy to no longer consider a beta
      • A v1.0.0 might be coming soon, but its just a version number, doesn't mean anything special
    • Changed default rdflib triplestore backend to 'Memory2' as above.
    • Tiny optimisations in the way sh:message items are added to a validation report graph.

    Fixed

    • Regression since v0.11.0, sh:value and sh:focusNode from the datagraph were not included in the validation report graph if the datagraph was of type rdflib.ConjunctiveGraph or rdflib.Dataset.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.12.1-py3-none-any.whl(115.15 KB)
    pyshacl-0.12.1.tar.gz(148.18 KB)
    pyshacl.exe(11.24 MB)
  • v0.12.0(Jul 10, 2020)

    Removed

    Announcement

    • Python 3.5 support is removed. PySHACL now requires Python 3.6 or above.
      • Routine tests are run using Python 3.6.11, 3.7.8, and 3.8.2.
      • Python 3.9 might work but is not yet supported.

    Added

    • Python 3.6-compatible type hinting is added throughout the codebase
    • MyPy library is used to run type checking during testing process
    • Flake8 linting is added to enforce PEP8
    • isort is added to enforce imports linting
    • Black is added to keep formatting consistent across releases

    Changed

    • PySHACL is no longer a setuptools-based project with a setup.py and requirements.txt file.
    • PySHACL is now a PEP518 & PEP517 project, it uses pyproject.toml and poetry to manage dependencies, build and install.
    • For best compatibility when installing from PyPI with pip, upgrade to pip v18.1.0 or above.
      • If you're on Ubuntu 16.04 or 18.04, you will need to run sudo pip3 install --upgrade pip
    • Editor Line Length for PySHACL code is now set to 119 as opposed to 79 chars.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.12.0-py3-none-any.whl(109.88 KB)
    pyshacl-0.12.0.tar.gz(143.65 KB)
    pyshacl.exe(11.24 MB)
  • v0.11.6.post1(Jul 9, 2020)

    Added

    • New feature to CLI tool
      • -V shows the PySHACL version
    • Run module directly
      • You can get access to the same CLI tool if you install the module and run it using python3 -m pyshacl
      • See python3 -m pyshacl --help for more details

    Announcement

    • This is the final version with Python v3.5 support
      • Versions 0.12.0 and above will have newer package management and dependency management, and will require Python v3.6+.
    Source code(tar.gz)
    Source code(zip)
    pyshacl-0.11.6.post1-py3-none-any.whl(82.69 KB)
    pyshacl-0.11.6.post1.tar.gz(133.79 KB)
    pyshacl.exe(10.71 MB)
Owner
RDFLib
RDFlib, the GitHub organization, is a volunteer-maintained collection of Python tools for working with RDF data.
RDFLib
Python Eacc is a minimalist but flexible Lexer/Parser tool in Python.

Python Eacc is a parsing tool it implements a flexible lexer and a straightforward approach to analyze documents.

Iury de oliveira gomes figueiredo 60 Nov 16, 2022
Repository for learning Python (Python Tutorial)

Repository for learning Python (Python Tutorial) Languages and Tools ?? Overview ?? Repository for learning Python (Python Tutorial) Languages and Too

Swiftman 2 Aug 22, 2022
A python package to avoid writing and maintaining duplicated python docstrings.

docstring-inheritance is a python package to avoid writing and maintaining duplicated python docstrings.

Antoine Dechaume 15 Dec 7, 2022
advance python series: Data Classes, OOPs, python

Working With Pydantic - Built-in Data Process ========================== Normal way to process data (reading json file): the normal princiople, it's f

Phung Hưng Binh 1 Nov 8, 2021
A comprehensive and FREE Online Python Development tutorial going step-by-step into the world of Python.

FREE Reverse Engineering Self-Study Course HERE Fundamental Python The book and code repo for the FREE Fundamental Python book by Kevin Thomas. FREE B

Kevin Thomas 7 Mar 19, 2022
A simple USI Shogi Engine written in python using python-shogi.

Revengeshogi My attempt at creating a USI Shogi Engine in python using python-shogi. Current State of Engine Currently only generating random moves us

null 1 Jan 6, 2022
Python-slp - Side Ledger Protocol With Python

Side Ledger Protocol Run python-slp node First install Mongo DB and run the mong

Solar 3 Mar 2, 2022
Python-samples - This project is to help someone need some practices when learning python language

Python-samples - This project is to help someone need some practices when learning python language

Gui Chen 0 Feb 14, 2022
Valentine-with-Python - A Python program generates an animation of a heart with cool texts of your loved one

Valentine with Python Valentines with Python is a mini fun project I have coded.

Niraj Tiwari 4 Dec 31, 2022
A curated list of awesome tools for Sphinx Python Documentation Generator

Awesome Sphinx (Python Documentation Generator) A curated list of awesome extra libraries, software and resources for Sphinx (Python Documentation Gen

Hyunjun Kim 831 Dec 27, 2022
API Documentation for Python Projects

API Documentation for Python Projects. Example pdoc -o ./html pdoc generates this website: pdoc.dev/docs. Installation pip install pdoc pdoc is compat

mitmproxy 1.4k Jan 7, 2023
🏆 A ranked list of awesome python developer tools and libraries. Updated weekly.

Best-of Python Developer Tools ?? A ranked list of awesome python developer tools and libraries. Updated weekly. This curated list contains 250 awesom

Machine Learning Tooling 646 Jan 7, 2023
Run `black` on python code blocks in documentation files

blacken-docs Run black on python code blocks in documentation files. install pip install blacken-docs usage blacken-docs provides a single executable

Anthony Sottile 460 Dec 23, 2022
Legacy python processor for AsciiDoc

AsciiDoc.py This branch is tracking the alpha, in-progress 10.x release. For the stable 9.x code, please go to the 9.x branch! AsciiDoc is a text docu

AsciiDoc.py 178 Dec 25, 2022
📖 Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.

lazydocs Generate markdown API documentation for Google-style Python docstring. Getting Started • Features • Documentation • Support • Contribution •

Machine Learning Tooling 118 Dec 31, 2022
A Python module for creating Excel XLSX files.

XlsxWriter XlsxWriter is a Python module for writing files in the Excel 2007+ XLSX file format. XlsxWriter can be used to write text, numbers, formula

John McNamara 3.1k Dec 29, 2022
Materi workshop "Light up your Python!" Himpunan Mahasiswa Sistem Informasi Fakultas Ilmu Komputer Universitas Singaperbangsa Karawang, 4 September 2021 (Online via Zoom).

Workshop Python UNSIKA 2021 Materi workshop "Light up your Python!" Himpunan Mahasiswa Sistem Informasi Fakultas Ilmu Komputer Universitas Singaperban

Eka Putra 20 Mar 24, 2022
Exercism exercises in Python.

Exercism exercises in Python.

Exercism 1.3k Jan 4, 2023
This is a repository for "100 days of code challenge" projects. You can reach all projects from beginner to professional which are written in Python.

100 Days of Code It's a challenge that aims to gain code practice and enhance programming knowledge. Day #1 Create a Band Name Generator It's actually

SelenNB 2 May 12, 2022