JSON Schema validation library

Overview

jsonschema

ci codecov Crates.io docs.rs gitter

A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible.

Supported drafts:

  • Draft 7 (except optional idn-hostname.json test case)
  • Draft 6
  • Draft 4 (except optional bignum.json test case)
# Cargo.toml
jsonschema = "0.13"

To validate documents against some schema and get validation errors (if any):

use jsonschema::JSONSchema;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    let compiled = JSONSchema::compile(&schema)
        .expect("A valid schema");
    let result = compiled.validate(&instance);
    if let Err(errors) = result {
        for error in errors {
            println!("Validation error: {}", error);
            println!(
                "Instance path: {}", error.instance_path
            );
        }
    }
}

Each error has an instance_path attribute that indicates the path to the erroneous part within the validated instance. It could be transformed to JSON Pointer via .to_string() or to Vec via .into_vec().

If you only need to know whether document is valid or not (which is faster):

use jsonschema::is_valid;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    assert!(is_valid(&schema, &instance));
}

Or use a compiled schema (preferred):

use jsonschema::JSONSchema;
use serde_json::json;

fn main() {
    let schema = json!({"maxLength": 5});
    let instance = json!("foo");
    // Draft is detected automatically
    // with fallback to Draft7
    let compiled = JSONSchema::compile(&schema)
        .expect("A valid schema");
    assert!(compiled.is_valid(&instance));
}

Output styles

jsonschema supports basic & flag output styles from Draft 2019-09, so you can serialize the validation results with serde:

use jsonschema::{Output, BasicOutput, JSONSchema};

fn main() {
    let schema_json = serde_json::json!({
        "title": "string value",
        "type": "string"
    });
    let instance = serde_json::json!{"some string"};
    let schema = JSONSchema::options()
        .compile(&schema_json)
        .expect("A valid schema");
    
    let output: BasicOutput = schema.apply(&instance).basic();
    let output_json = serde_json::to_value(output)
        .expect("Failed to serialize output");
    
    assert_eq!(
        output_json, 
        serde_json::json!({
            "valid": true,
            "annotations": [
                {
                    "keywordLocation": "",
                    "instanceLocation": "",
                    "annotations": {
                        "title": "string value"
                    }
                }
            ]
        })
    );
}

Status

This library is functional and ready for use, but its API is still evolving to the 1.0 API.

Bindings

  • Python - See the ./bindings/python directory
  • Ruby - a crate by @driv3r
  • NodeJS - a package by @ahungrynoob

Performance

There is a comparison with other JSON Schema validators written in Rust - jsonschema_valid==0.4.0 and valico==3.6.0.

Test machine i8700K (12 cores), 32GB RAM.

Input values and schemas:

Case Schema size Instance size
OpenAPI 18 KB 4.5 MB
Swagger 25 KB 3.0 MB
Canada 4.8 KB 2.1 MB
CITM catalog 2.3 KB 501 KB
Fast (valid) 595 B 55 B
Fast (invalid) 595 B 60 B

Here is the average time for each contender to validate. Ratios are given against compiled JSONSchema using its validate method. The is_valid method is faster, but gives only a boolean return value:

Case jsonschema_valid valico jsonschema (validate) jsonschema (is_valid)
OpenAPI - (1) - (1) 4.717 ms 4.279 ms (x0.90)
Swagger - (2) 83.357 ms (x12.47) 6.681 ms 4.533 ms (x0.67)
Canada 32.987 ms (x31.38) 141.41 ms (x134.54) 1.051 ms 1.046 ms (x0.99)
CITM catalog 4.735 ms (x2.00) 13.222 ms (x5.58) 2.367 ms 535.07 us (x0.22)
Fast (valid) 2.00 us (x3.85) 3.18 us (x6.13) 518.39 ns 97.91 ns (x0.18)
Fast (invalid) 339.28 ns (x0.50) 3.34 us (x5.00) 667.55 ns 5.41ns (x0.01)

Notes:

  1. jsonschema_valid and valico do not handle valid path instances matching the ^\\/ regex.

  2. jsonschema_valid fails to resolve local references (e.g. #/definitions/definitions).

You can find benchmark code in benches/jsonschema.rs, Rust version is 1.57.

Support

If you have anything to discuss regarding this library, please, join our gitter!

Comments
  • Add a `path` field to `ValidationError`

    Add a `path` field to `ValidationError`

    Summary

    On validation error make the path to the problematic field available to the client. Currently the path is only a field in ValidationError but later it can be used to improve the error messages (related to #100, #144)

    Example

    Schema
    {
       "type":"object",
       "properties":{
          "foo":{
             "type":"object",
             "properties":{
                "bar":{
                   "type":"integer"
                }
             }
          }
       }
    }
    
    Input
    {
      "foo": {
        "bar": "some string"
      }
    }
    
    Produced Error

    path: [foo, bar]

    opened by gavadinov 16
  • patternProperties interferes with properties

    patternProperties interferes with properties

    It seems like the presence of "patternProperties" interferes with validation that should happen in "properties".

    This replit shows an example of an instance that should fail validation (and does fail in jsonschemavalidator.net). If you comment out the "patternProperties" portion of the schema, it fails as expected.

    I don't have a good enough understanding of the jsonschema internals, but it seems like this could be related to changes from #173 .

    UPDATE: I updated the replit and the jsonschemavalidator link to provide a more minimal example.

    opened by duckontheweb 16
  • [FEATURE] User-defined validation for the `format` keyword

    [FEATURE] User-defined validation for the `format` keyword

    At the moment, jsonschema supports the format keyword for formats defined in Drafts 4, 6, 7. However, JSON Schema allows custom values for the format keyword.

    It will be awesome to provide a way for the end-user to validate their own formats.

    As all validators are "compiled" to some structs (to avoid some run-time overhead e.g., map lookup, etc.), I think that the public API might look like this:

    use jsonschema;
    use std::collections::HashMap;
    use serde_json::{json, Value};
    
    struct CustomFormatValidator {}
    
    // New trait somehow derived from `Validate`?
    impl jsonschema::SomeNewTrait for CustomFormatValidator {
        fn is_valid(&self, _: &jsonschema::JSONSchema, instance: &Value) -> bool {
            if let Value::String(item) = instance {
                item == "bar"  // Format matching logic goes here
            } else {
                true
            }
        }
    }
    
    fn main() -> Result<(), jsonschema::CompilationError> {
        let schema = json!({"format": "custom"});
        let instance = json!("foo");
        let mut formats = HashMap::new();
        formats.insert(
            "custom": CustomFormatValidator
        );
        // Shortcuts
        jsonschema::is_valid(&schema, &instance, Some(formats))
        // Compiled schema
        let compiled = jsonschema::JSONSchema::options()
            .formats(formats)
            .compile(&schema)
            .expect("Invalid schema");
        compiled.is_valid(&instance)
    }
    

    It will be nice to avoid the boilerplate that extracts the actual string from Value::String, so the user won't write this code for each format.

    Anticipated changes to implement this:

    • Take extra argument for the formats map in jsonschema::is_valid;
    • Extend CompilationOptions builder to handle formats;
    • Store the formats map as a part of CompilationContext;
    • Inside format.rs take a look at this mapping before going to take a look at the built-in validators;
    • Make the Validate trait public and implement some trait on top of it to avoid boilerplate?

    For now, I think it is OK to keep the scope on the Rust code only. A similar feature for bindings could be implemented later.

    help wanted good first issue Priority: Medium Type: Feature Difficulty: Medium 
    opened by Stranger6667 16
  • HTTP Resolver Not Working?

    HTTP Resolver Not Working?

    Hi,

    Sorry, I've been trying to work this out myself, but I'm not sure what's going wrong.

    Validation error: failed to resolve https://schema.rawkode.dev/organization: error sending request for url (https://schema.rawkode.dev/organization): error trying to connect: invalid URL, scheme is not http
    Instance path:
    Validation error: failed to resolve https://schema.rawkode.dev/person: error sending request for url (https://schema.rawkode.dev/person): error trying to connect: invalid URL, scheme is not http
    Instance path:
    

    These URLs are definitely OK and I'm not sure what I've done wrong.

    I've tried using http instead of https in my schema, but the problem is the same.

    Any help appreciated.

    Priority: High Type: Bug 
    opened by rawkode 15
  • Allow Schema Value to be Owned

    Allow Schema Value to be Owned

    In the current form it is impossible to retain a non-static reference to JSONSchema until the end of the program's execution. The below block demonstrates the problem.

    #[actix_web::main]
    async fn main() -> std::io::Result<()> {
        let schema_file = fs::File::open("config/schema.json").unwrap();
    
        let schema: Value = serde_json::from_reader(schema_file).unwrap();
        let validator = JSONSchema::compile(&schema, None).unwrap();
    
        let ad = AppData{
            validator: Arc::new(validator)
        };
    
        HttpServer::new(move ||{
            App::new()
                .data(ad.clone())
                .service(echo)
        })
        .bind("localhost:9070")?
        .run()
        .await
    }
    
    #[derive(Clone)]
    pub struct AppData<'a> {
        pub validator: Arc<JSONSchema<'a>>
    }
    

    The above code results in the below compiler error:

    error[E0597]: `schema` does not live long enough
      --> src/main.rs:15:41
       |
    15 |     let validator = JSONSchema::compile(&schema, None).unwrap();
       |                     --------------------^^^^^^^-------
       |                     |                   |
       |                     |                   borrowed value does not live long enough
       |                     argument requires that `schema` is borrowed for `'static`
    ...
    31 | }
       | - `schema` dropped here while still borrowed
    

    One(or the only?) way to get this fixed is to let JSONSchema own the Value containing schema.

    opened by kayyagari 12
  • fix: convert python tuples into lists

    fix: convert python tuples into lists

    We use tuples for geographic coordinates and bounding boxes which doesn't seem to have a direct mapping in JSON Schema.

    This pull requests converts the tuple into a list to fix ValueError: Unsupported type: 'tuple' when validating a object that contains a tuple.

    { "bbox": (1,2,3,4) }
    
    opened by blacha 11
  • Add support for look-around patterns using fancy-regex

    Add support for look-around patterns using fancy-regex

    Uses fancy-regex in place of regex for matching against all patterns in JSON schemas.

    regex is still used when converting a RegEx for ECMA 262 to take advantage of the Regex::replace_all method, which is not included in fancy-regex. Since fancy-regex depends on regex anyway, this does not affect the final size of the compiled binary (thanks to this commit for the idea).

    I also removed the test from #214, since that pattern is supported in this PR.

    opened by duckontheweb 10
  • Report schema paths in validation errors

    Report schema paths in validation errors

    When a validation error is reported, it contains only instance_path that points to the erroneous part of the input, but there is no information about what part of the schema caused this error.

    The gist of the implementation is to keep a stack of strings while the input schema is compiled. The stack should be convertible to paths::JSONPointer.

    The process for regular, non-composite validators (more on this below):

    • When the compilation process enters a validator, push its relevant keyword name to the stack (e.g. properties);
    • When the validator struct is created, clone this stack and store it within this struct;
    • When the validator is compiled - pop from the stack

    I think that pushing/popping could be implemented similarly to instance_path - a linked list without heap allocations. This way, the actual popping step is not needed at all.

    Composite validators are slightly more complicated - they are single structs that implement the logic for multiple different keywords or their specific variants (e.g. this one handles patternProperties and additionalProperties: false) which is done due to performance reasons.

    To handle them, the stack should not be pushed during compilation, but rather during validation - if a validation error happens, the stored stack should be cloned, and the exact keyword variant should be pushed to it. And finally, it is moved to ValidationError.

    Validators live in keywords.

    A lot of work in this direction was done by @gavadinov in https://github.com/gavadinov/jsonschema-rs/commit/40149322bfdf5549a0150122a00fa66dfe770828, and I think that it could be a great base for implementing this feature.

    @gavadinov, let me know if you are OK if somebody can base their work on your commit, or if you want to continue it yourself :)

    help wanted good first issue Priority: High Type: Feature Difficulty: Medium 
    opened by Stranger6667 10
  • Replace `ValidationError::schema` with custom errors

    Replace `ValidationError::schema` with custom errors

    Hi there, bump into here via hacktoberfest mark.

    Honestly most likely it doesn't address JSONPointer correctly in errors. And I was not sure which type of error better use for things like as64 error or regex syntax error but I did used ValidationError::format.

    Note: there might be an issue with cargo fmt.

    closes #235

    Thank you for the library.

    opened by zhiburt 9
  • Use the original value in error formatting for some numeric validators

    Use the original value in error formatting for some numeric validators

    Validating 3.0 against {"exclusiveMaximum": 3.0} will give 3.0 is greater than or equal to the maximum of 3. But the last number in the error message should be 3.0. It affects exclusiveMaximum, exclusiveMinimum, minimum, maximum.

    Priority: Low Type: Bug Difficulty: Easy 
    opened by Stranger6667 9
  • Use a better structure instead of `Map` in validators

    Use a better structure instead of `Map` in validators

    E.g. in AdditionalPropertiesNotEmptyValidator. Because during the validation we use self.properties.contains_key(*property) which will be O(logN) (because of BTreeMap). We can use HashMap to have O(1).

    As perf showed, significant time is spent on it

    Priority: Low Type: Enhancement Topic: Performance 
    opened by Stranger6667 8
  • Don't depend on a particular ahash rng

    Don't depend on a particular ahash rng

    I'm not very familiar with ahash, or WASM in general, but it seems like ahash's default randomness source panics when run inside a WASM environment. This change should make it so that jsonschema uses the runtime randomness source by default, but will not require it when included with default-features = false.

    opened by SamWilsn 3
  • Try compile tauri in wasm target , encountered some problems

    Try compile tauri in wasm target , encountered some problems

    Hi, First of all, I'm not the maintainer of tauri.

    I find the tauri-cli crate used jsonschema, and I am trying run it with wasm , When I run here, an error is reported.

    so, I changed it like this, Because tauri does not use this part when parsing json. From the perspective of jsonschema, this change should be inappropriate. Is there a more reasonable solution ?

    opened by tu6ge 0
  • multipleOf logic produces incorrect results

    multipleOf logic produces incorrect results

    The logic for multipleOf compares the remainder of a division with the f64 EPSILON constant to decide if a number is a multiple or not. This approach doesn't make sense. EPSILON is simply the difference between 1.0 and the next larger representable number. With regards to the multipleOf calculation this is just an arbitrary constant and comparing remainders to it gives us no guarantees.

    EG:

    1e-15 is a multiple of 1e-16. But (1e-15 / 1e-16) % 1.0 = 1.7763568394002505e-15 which is greater than epsilon so it is incorrectly said to not be a multiple.

    1.3e-16 is not a multiple of 1.3. But (1.3e-16 / 1.3) % 1.0 = 9.999999999999999e-17 which is less than epsilon so it is incorrectly said to be a multiple.

    opened by DanielBauman88 1
  • feat: Custom keyword validation

    feat: Custom keyword validation

    Add support for user defined custom keyword validation. The user provides and registers custom validator functions when configuring a JSONSchema.

    Custom keyword validators may be used when the user wants to enforce constraints that can't, or can't easily, be expressed in JSON schema.

    This PR addresses at least some of the features requested in #379 by @tamasfe as well as my own. The same pros and cons described in #379 apply. The PR also follows the structure of #354, but of course uses validation functions as opposed to sub-schema for validation.

    We could collapse the two user implementable validation functions into a single trait, mirroring the internal Validate trait. Implementations would have to sit behind an Arc.

    Example usage

    use jsonschema::{CustomKeywordDefinition, ErrorIterator, JSONSchema, paths::JSONPointer};
    use serde_json::{json, Value};
    use std::sync::Arc;
    
    fn validate(
        instance: &Value,
        instance_path: JSONPointer,
        schema: Arc<Value>,
        schema_path: JSONPointer,
    ) -> ErrorIterator {
        // ... validate instance ... 
        Box::new(None.into_iter())
    }
    fn is_valid(instance: &Value, schema: &Value) -> bool {
        // ... determine if instance is valid ... 
        return true;
    }
    let definition = CustomKeywordDefinition::Validator { validate, is_valid };
    assert!(JSONSchema::options()
        .with_custom_keyword("my-type", definition)
        .compile(&json!({ "my-type": "my-schema"}))
        .expect("A valid schema")
        .is_valid(&json!({ "a": "b"})));
    
    opened by bnjt 8
  • Async SchemaResolver

    Async SchemaResolver

    I'm using an async S3 client to load schemas from a bucket. I've hit a wall in that SchemaResolver is not async, but I need to call an async method within it. Is AsyncSchemaResolver on the roadmap or is there a known work around?

    ///
    /// Download an object from the S3 bucket and turn the result as Bytes
    /// 
    pub async fn download_object(client: &Client, bucket_name: &str, key: &str) -> Result<Bytes, S3Error> {
        let resp = client
            .get_object()
            .bucket(bucket_name)
            .key(key)
            .send()
            .await?;
        let data = match resp.body.collect().await {
            Ok(data) => data,
            Err(e) => return Err(S3Error::GetObjectError{ bucket_name: bucket_name.to_string(), key: key.to_string(), msg: e.to_string() }),
        };
        let data = data.into_bytes();
    
        Ok(data)
    }
    
    ///
    /// Download and object from the S3 bucket and return the results as a json Value
    /// 
    pub async fn download_json(client: &Client, bucket_name: &str, path: &str) -> Result<serde_json::Value, JsonError> {
        let data = download_object(client, bucket_name, path).await?;
        let json: serde_json::Value = serde_json::from_slice(&data)?;
        Ok(json)
    }
    
    struct S3SchemaResolver {
        client: Client,         // S3 client
        bucket_name: String     // name of bucket that holds schemas
    }
    
    impl SchemaResolver for S3SchemaResolver {
        fn resolve(&self, root_schema: &Value, url: &Url, original_reference: &str) -> Result<Arc<Value>, SchemaResolverError> {
            //
            // add the base path to the reference
            //
            match download_json(&self.client, &self.bucket_name, original_reference).await {
                Ok(schema_value) => Ok(Arc::new(schema_value)),
                Err(json_error) => {
                    let msg = format!("{}", json_error);
                    log::error!("{}", msg);
                    Err(anyhow::anyhow!(msg))
                }
            }
        }
    }
    
    opened by Ezward 0
Releases(python-v0.16.0)
  • python-v0.16.0(May 12, 2022)

    Added

    • Python 3.10 support

    Fixed

    • Installation error due to pyo3-built incompatibility
    • Memory leak in iter_errors. #325

    Changed

    • Update pyo3 to 0.16.

    Removed

    • Support for Python 3.6
    Source code(tar.gz)
    Source code(zip)
  • rust-v0.16.0(Apr 21, 2022)

Owner
Dmitry Dygalo
Building a platform for effortless API testing
Dmitry Dygalo
Generate code from JSON schema files

json-schema-codegen Generate code from JSON schema files. Table of contents Introduction Currently supported languages Requirements Installation Usage

Daniele Esposti 30 Dec 23, 2022
Json utils is a python module that you can use when working with json files.

Json-utils Json utils is a python module that you can use when working with json files. it comes packed with a lot of featrues Features Converting jso

Advik 4 Apr 24, 2022
Random JSON Key:Pair Json Generator

Random JSON Key:Value Pair Generator This simple script take an engish dictionary of words and and makes random key value pairs. The dictionary has ap

Chris Edwards 1 Oct 14, 2021
With the help of json txt you can use your txt file as a json file in a very simple way

json txt With the help of json txt you can use your txt file as a json file in a very simple way Dependencies re filemod pip install filemod Installat

Kshitij 1 Dec 14, 2022
Same as json.dumps or json.loads, feapson support feapson.dumps and feapson.loads

Same as json.dumps or json.loads, feapson support feapson.dumps and feapson.loads

boris 5 Dec 1, 2021
Ibmi-json-beautify - Beautify json string with python

Ibmi-json-beautify - Beautify json string with python

Jefferson Vaughn 3 Feb 2, 2022
cysimdjson - Very fast Python JSON parsing library

Fast JSON parsing library for Python, 7-12 times faster than standard Python JSON parser.

TeskaLabs 235 Dec 29, 2022
A JSON utility library for Python featuring Django-style queries and mutations.

JSON Enhanced JSON Enhanced implements fast and pythonic queries and mutations for JSON objects. Installation You can install json-enhanced with pip:

Collisio Technologies 4 Aug 22, 2022
Simple Python Library to convert JSON to XML

json2xml Simple Python Library to convert JSON to XML

Vinit Kumar 79 Nov 11, 2022
A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

Adrian Edwards 9 Dec 2, 2022
JSON Interoperability Vulnerability Labs

JSON Interoperability Vulnerability Labs Description These are the companion labs to my research article "An Exploration of JSON Interoperability Vuln

Bishop Fox 168 Dec 25, 2022
A tools to find the path of a specific key in deep nested JSON.

如何快速从深层嵌套 JSON 中找到特定的 Key #公众号 在爬虫开发的过程中,我们经常遇到一些 Ajax 加载的接口会返回 JSON 数据。

kingname 56 Dec 13, 2022
simplejson is a simple, fast, extensible JSON encoder/decoder for Python

simplejson simplejson is a simple, fast, complete, correct and extensible JSON <http://json.org> encoder and decoder for Python 3.3+ with legacy suppo

null 1.5k Jan 5, 2023
A fast JSON parser/generator for C++ with both SAX/DOM style API

A fast JSON parser/generator for C++ with both SAX/DOM style API Tencent is pleased to support the open source community by making RapidJSON available

Tencent 12.6k Dec 30, 2022
simdjson : Parsing gigabytes of JSON per second

JSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh approach. The simdjson library uses commonly available SIMD instructions and microparallel algorithms to parse JSON 4x faster than RapidJSON and 25x faster than JSON for Modern C++.

null 16.3k Dec 29, 2022
import json files directly in your python scripts

Install Install from git repository pip install git+https://github.com/zaghaghi/direct-json-import.git Use With the following json in a file named inf

Hamed Zaghaghi 51 Dec 1, 2021
An tiny CLI to load data from a JSON File during development.

JSON Server - An tiny CLI to load data from a JSON File during development.

Yuvraj.M 4 Mar 22, 2022
Convert your subscriptions csv file into a valid json for Newpipe!

Newpipe-CSV-Fixer Convert your Google subscriptions CSV file into a valid JSON for Newpipe! Thanks to nikcorg for sharing how to convert the CSV into

Juanjo 44 Dec 29, 2022
Low code JSON to extract data in one line

JSON Inline Low code JSON to extract data in one line ENG RU Installation pip install json-inline Usage Rules Modificator Description ?key:value Searc

Aleksandr Sokolov 12 Mar 9, 2022