🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

Explosion

Last update: Dec 28, 2022

Related tags

Data Serialization python yaml serialization json msgpack python-3 pickle python-2 ujson

Overview

srsly: Modern high-performance serialization utilities for Python

This package bundles some of the best Python serialization libraries into one standalone package, with a high-level API that makes it easy to write code that's correct across platforms and Pythons. This allows us to provide all the serialization utilities we need in a single binary wheel. Currently supports JSON, JSONL, MessagePack, Pickle and YAML.

Motivation

Serialization is hard, especially across Python versions and multiple platforms. After dealing with many subtle bugs over the years (encodings, locales, large files) our libraries like spaCy and Prodigy have steadily grown a number of utility functions to wrap the multiple serialization formats we need to support (especially json, msgpack and pickle). These wrapping functions ended up duplicated across our codebases, so we wanted to put them in one place.

At the same time, we noticed that having a lot of small dependencies was making maintenance harder, and making installation slower. To solve this, we've made srsly standalone, by including the component packages directly within it. This way we can provide all the serialization utilities we need in a single binary wheel.

srsly currently includes forks of the following packages:

ujson
msgpack
msgpack-numpy
cloudpickle
ruamel.yaml (without unsafe implementations!)

Installation

⚠️ Note that v2.x is only compatible with Python 3.6+. For 2.7+ compatibility, use v1.x.

srsly can be installed from pip. Before installing, make sure that your pip, setuptools and wheel are up to date.

pip install -U pip setuptools wheel
pip install srsly

Or from conda via conda-forge:

conda install -c conda-forge srsly

Alternatively, you can also compile the library from source. You'll need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler (XCode command-line tools on macOS / OS X or Visual C++ build tools on Windows), pip, virtualenv and git installed.

pip install -r requirements.txt  # install development dependencies
python setup.py build_ext --inplace  # compile the library

API

JSON

📦 The underlying module is exposed via srsly.ujson. However, we normally interact with it via the utility functions only.

`function` `srsly.json_dumps`

Serialize an object to a JSON string. Falls back to json if sort_keys=True is used (until it's fixed in ujson).

data = {"foo": "bar", "baz": 123}
json_string = srsly.json_dumps(data)

Argument	Type	Description
`data`	-	The JSON-serializable data to output.
`indent`	int	Number of spaces used to indent JSON. Defaults to `0`.
`sort_keys`	bool	Sort dictionary keys. Defaults to `False`.
RETURNS	str	The serialized string.

`function` `srsly.json_loads`

Deserialize unicode or bytes to a Python object.

data = '{"foo": "bar", "baz": 123}'
obj = srsly.json_loads(data)

Argument	Type	Description
`data`	str / bytes	The data to deserialize.
RETURNS	-	The deserialized Python object.

`function` `srsly.write_json`

Create a JSON file and dump contents or write to standard output.

data = {"foo": "bar", "baz": 123}
srsly.write_json("/path/to/file.json", data)

Argument	Type	Description
`path`	str / `Path`	The file path or `"-"` to write to stdout.
`data`	-	The JSON-serializable data to output.
`indent`	int	Number of spaces used to indent JSON. Defaults to `2`.

`function` `srsly.read_json`

Load JSON from a file or standard input.

data = srsly.read_json("/path/to/file.json")

Argument	Type	Description
`path`	str / `Path`	The file path or `"-"` to read from stdin.
RETURNS	dict / list	The loaded JSON content.

`function` `srsly.write_gzip_json`

Create a gzipped JSON file and dump contents.

data = {"foo": "bar", "baz": 123}
srsly.write_gzip_json("/path/to/file.json.gz", data)

Argument	Type	Description
`path`	str / `Path`	The file path.
`data`	-	The JSON-serializable data to output.
`indent`	int	Number of spaces used to indent JSON. Defaults to `2`.

`function` `srsly.read_gzip_json`

Load gzipped JSON from a file.

data = srsly.read_gzip_json("/path/to/file.json.gz")

Argument	Type	Description
`path`	str / `Path`	The file path.
RETURNS	dict / list	The loaded JSON content.

`function` `srsly.write_jsonl`

Create a JSONL file (newline-delimited JSON) and dump contents line by line, or write to standard output.

data = [{"foo": "bar"}, {"baz": 123}]
srsly.write_jsonl("/path/to/file.jsonl", data)

Argument	Type	Description
`path`	str / `Path`	The file path or `"-"` to write to stdout.
`lines`	iterable	The JSON-serializable lines.
`append`	bool	Append to an existing file. Will open it in `"a"` mode and insert a newline before writing lines. Defaults to `False`.
`append_new_line`	bool	Defines whether a new line should first be written when appending to an existing file. Defaults to `True`.

`function` `srsly.read_jsonl`

Read a JSONL file (newline-delimited JSON) or from JSONL data from standard input and yield contents line by line. Blank lines will always be skipped.

data = srsly.read_jsonl("/path/to/file.jsonl")

Argument	Type	Description
`path`	str / Path	The file path or `"-"` to read from stdin.
`skip`	bool	Skip broken lines and don't raise `ValueError`. Defaults to `False`.
YIELDS	-	The loaded JSON contents of each line.

`function` `srsly.is_json_serializable`

Check if a Python object is JSON-serializable.

assert srsly.is_json_serializable({"hello": "world"}) is True
assert srsly.is_json_serializable(lambda x: x) is False

Argument	Type	Description
`obj`	-	The object to check.
RETURNS	bool	Whether the object is JSON-serializable.

msgpack

📦 The underlying module is exposed via srsly.msgpack. However, we normally interact with it via the utility functions only.

`function` `srsly.msgpack_dumps`

Serialize an object to a msgpack byte string.

data = {"foo": "bar", "baz": 123}
msg = srsly.msgpack_dumps(data)

Argument	Type	Description
`data`	-	The data to serialize.
RETURNS	bytes	The serialized bytes.

`function` `srsly.msgpack_loads`

Deserialize msgpack bytes to a Python object.

msg = b"\x82\xa3foo\xa3bar\xa3baz{"
data = srsly.msgpack_loads(msg)

Argument	Type	Description
`data`	bytes	The data to deserialize.
`use_list`	bool	Don't use tuples instead of lists. Can make deserialization slower. Defaults to `True`.
RETURNS	-	The deserialized Python object.

`function` `srsly.write_msgpack`

Create a msgpack file and dump contents.

data = {"foo": "bar", "baz": 123}
srsly.write_msgpack("/path/to/file.msg", data)

Argument	Type	Description
`path`	str / `Path`	The file path.
`data`	-	The data to serialize.

`function` `srsly.read_msgpack`

Load a msgpack file.

data = srsly.read_msgpack("/path/to/file.msg")

Argument	Type	Description
`path`	str / `Path`	The file path.
`use_list`	bool	Don't use tuples instead of lists. Can make deserialization slower. Defaults to `True`.
RETURNS	-	The loaded and deserialized content.

pickle

📦 The underlying module is exposed via srsly.cloudpickle. However, we normally interact with it via the utility functions only.

`function` `srsly.pickle_dumps`

Serialize a Python object with pickle.

data = {"foo": "bar", "baz": 123}
pickled_data = srsly.pickle_dumps(data)

Argument	Type	Description
`data`	-	The object to serialize.
`protocol`	int	Protocol to use. `-1` for highest. Defaults to `None`.
RETURNS	bytes	The serialized object.

`function` `srsly.pickle_loads`

Deserialize bytes with pickle.

pickled_data = b"\x80\x04\x95\x19\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x03foo\x94\x8c\x03bar\x94\x8c\x03baz\x94K{u."
data = srsly.pickle_loads(pickled_data)

Argument	Type	Description
`data`	bytes	The data to deserialize.
RETURNS	-	The deserialized Python object.

YAML

📦 The underlying module is exposed via srsly.ruamel_yaml. However, we normally interact with it via the utility functions only.

`function` `srsly.yaml_dumps`

Serialize an object to a YAML string. See the ruamel.yaml docs for details on the indentation format.

data = {"foo": "bar", "baz": 123}
yaml_string = srsly.yaml_dumps(data)

Argument	Type	Description
`data`	-	The JSON-serializable data to output.
`indent_mapping`	int	Mapping indentation. Defaults to `2`.
`indent_sequence`	int	Sequence indentation. Defaults to `4`.
`indent_offset`	int	Indentation offset. Defaults to `2`.
`sort_keys`	bool	Sort dictionary keys. Defaults to `False`.
RETURNS	str	The serialized string.

`function` `srsly.yaml_loads`

Deserialize unicode or a file object to a Python object.

data = 'foo: bar\nbaz: 123'
obj = srsly.yaml_loads(data)

Argument	Type	Description
`data`	str / file	The data to deserialize.
RETURNS	-	The deserialized Python object.

`function` `srsly.write_yaml`

Create a YAML file and dump contents or write to standard output.

data = {"foo": "bar", "baz": 123}
srsly.write_yaml("/path/to/file.yml", data)

Argument	Type	Description
`path`	str / `Path`	The file path or `"-"` to write to stdout.
`data`	-	The JSON-serializable data to output.
`indent_mapping`	int	Mapping indentation. Defaults to `2`.
`indent_sequence`	int	Sequence indentation. Defaults to `4`.
`indent_offset`	int	Indentation offset. Defaults to `2`.
`sort_keys`	bool	Sort dictionary keys. Defaults to `False`.

`function` `srsly.read_yaml`

Load YAML from a file or standard input.

data = srsly.read_yaml("/path/to/file.yml")

Argument	Type	Description
`path`	str / `Path`	The file path or `"-"` to read from stdin.
RETURNS	dict / list	The loaded YAML content.

`function` `srsly.is_yaml_serializable`

Check if a Python object is YAML-serializable.

assert srsly.is_yaml_serializable({"hello": "world"}) is True
assert srsly.is_yaml_serializable(lambda x: x) is False

Argument	Type	Description
`obj`	-	The object to check.
RETURNS	bool	Whether the object is YAML-serializable.

Comments

No module named 'srsly.ujson.ujson'

I'm installing srsly on an AWS lambda but I keep getting an error : No module named 'srsly.ujson.ujson'

It's install with srsly inside its package but can't be access. I tried to change all declaration to a global import ujson which i downloaded without any success ...

Any ideas ?

PS : I'm installing a Spacy lambda which require srsly

opened by antoineGit 8
Fix Typing for `JSONOutput` and `JSONOutputBin`
Description

Add Tuple[Any, ...] for JSONOutput and JSONOutputBin. Otherwise type hinting will indicate mismatching types if tuples are an expected type, e.g. for values: Tuple[Any, ...] = srsly.load(value).

Also fixes CI image for Linux with Python 3.6.

Related to https://github.com/explosion/srsly/pull/79.

Types of change

Bug fix.

Checklist

[x] I confirm that I have the right to submit this contribution under the project's MIT license.

[x] I ran the tests, and all new and existing tests passed.

[x] My changes don't require a change to the documentation, or if they do, I've added all required information.

bug
opened by rmitsch 5
Update UltraJSON

Hi,

UltraJSON was updated again to fix a security vulnerability with invalid surrogate pair characters.

Is it possible to merge the 555 and 550 changes associated with the 0.5.4 release into srsly?

Corey

opened by jhe921 4
Update Ultrajson

Hi,

Ultrajson recently got updated, I'm uncertain how it works with forks and srsly being standalone but is it possible to update the version used in srsly to use this latest version?

//Tim

opened by AsymptoticBrain 3
How to use `srsly.msgpack_dumps` with my custom class?

I want to serialize my custom class with srsly.msgpack_dumps, because it is stored in spacy.Doc. In other words, doc.to_disk fails because my custom class cannotn be serialized with srsly.msgpack_dumps. How to make my custom class to be able to save?

opened by tamuhey 3

Memory leaks in ujson

We've replaced the usage of json with srsly.ujson a while ago for that free performance boost since we are doing lots of JSON encoding/decoding and we already have it installed as part of spacy, but now we had to move back because of some terrible memory leaks:

import json
import random
import string
import psutil
from srsly import ujson as json

sample = lambda x: ''.join(
  random.choice(string.ascii_uppercase + string.digits) for _ in range(x))

process = psutil.Process()

for i in range(10):
  data = json.dumps({sample(99): sample(100000) for k in range(50)})
  json.loads(data)
  print(process.memory_info())

Output with ujson:

pmem(rss=24203264, vms=4400664576, pfaults=19049, pageins=0)
pmem(rss=29409280, vms=4414173184, pfaults=33898, pageins=0)
pmem(rss=34557952, vms=4419309568, pfaults=47960, pageins=0)
pmem(rss=39714816, vms=4424429568, pfaults=62855, pageins=0)
pmem(rss=44838912, vms=4429549568, pfaults=77571, pageins=0)
pmem(rss=50081792, vms=4434751488, pfaults=92307, pageins=0)
pmem(rss=55312384, vms=4439973888, pfaults=107288, pageins=0)
pmem(rss=60440576, vms=4445093888, pfaults=122711, pageins=0)
pmem(rss=65806336, vms=4451422208, pfaults=137578, pageins=0)
pmem(rss=70934528, vms=4456542208, pfaults=151382, pageins=0)

Output with stdlib json:

pmem(rss=17154048, vms=4385366016, pfaults=17692, pageins=0)
pmem(rss=17317888, vms=4403191808, pfaults=32047, pageins=0)
pmem(rss=17317888, vms=4403191808, pfaults=46541, pageins=0)
pmem(rss=17317888, vms=4403191808, pfaults=61035, pageins=0)
pmem(rss=17358848, vms=4403191808, pfaults=75539, pageins=0)
pmem(rss=17383424, vms=4403191808, pfaults=90039, pageins=0)
pmem(rss=17383424, vms=4403191808, pfaults=104533, pageins=0)
pmem(rss=17420288, vms=4403191808, pfaults=119036, pageins=0)
pmem(rss=17420288, vms=4403191808, pfaults=133530, pageins=0)
pmem(rss=17420288, vms=4403191808, pfaults=148024, pageins=0)

Benchmark ran on python3.7 on macos, but same leak exists on Debian with python27. You can increase the range from 10 and you will eventually run of our memory.

Days were spent on this issue, because I would never suspect the JSON library to be at fault, but it is. I don't know if it affects Spacy in any way.

Could be related: https://github.com/esnme/ultrajson/pull/270

bug

opened by sadovnychyi 3

Wordwises works, X-Ray doesn't
Wordwise seems to run fine, X-Ray errors :

Traceback (most recent call last): File "calibre\gui2\threaded_jobs.py", line 83, in start_work File "calibre_plugins.worddumb.parse_job", line 37, in do_job File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\spacy_3.1.1_3.8\spacy\__init__.py", line 11, in <module> File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\thinc_8.0.8_3.8\thinc\__init__.py", line 5, in <module> File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\thinc_8.0.8_3.8\thinc\config.py", line 14, in <module> File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\srsly_2.4.1_3.8\srsly\__init__.py", line 1, in <module> File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\srsly_2.4.1_3.8\srsly\_json_api.py", line 6, in <module> File "C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\srsly_2.4.1_3.8\srsly\ujson\__init__.py", line 1, in <module> ModuleNotFoundError: No module named 'srsly.ujson.ujson'

pip says srsly is installed correctly, and C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\srsly_2.4.1_3.8\srsly\ujson exists and has ujson.c inside among other files..

(unrelated? : I had to manually add blis_0.7.4_3.8 to the worddumb-libs on Windows 10 to clear an other error)

C:\Users\USERNAME\AppData\Roaming\calibre\plugins\worddumb-libs\srsly_2.4.1_3.8\srsly\ujson_init_.py line 1 is :

from .ujson import decode, encode, dump, dumps, load, loads # noqa: F401

Called with args: ((1587, 'MOBI', 'BBGL0Z779A', 'book.mobi', <calibre.ebooks.metadata.book.base.Metadata object at 0x082C44C0>, {'spacy': 'en_core_web_', 'wiki': 'en'}), False, True) {'notifications': <queue.Queue object at 0x082C46B8>, 'abort': <threading.Event object at 0x082C45F8>, 'log': <calibre.utils.logging.GUILog object at 0x082C45C8>}

Windows 10 Python 3.9 Calibre - 5.24

Installed the plugin today.
opened by benninkcorien 2
Bug: Cannot install if Cython is not already installed
Summary

The latest version 1 tag (v1.0.5) altered setup.py to directly import from Cython. This means that install fails with No module named 'Cython' unless you have manually installed it.

Projects that use srsly as a dependency will need to be edited to manually install Cython now, whereas before setuptools handed this install.

This also effects version 2 and the master branch.

Edit

I've looked into this and I think the expectation is that pyproject.toml will handle this cython pre-install. I am using python 3.6.5, I suspect this python version (or its setuptools) is not reading this file.

Reproduce

$ docker run -w /home/circleci circleci/python:3.6.5 bash -c "python3 -m venv venv; . venv/bin/activate; pip install srsly==1.0.5"

Collecting srsly==1.0.5 Downloading https://files.pythonhosted.org/packages/c7/08/abe935f33b69a08d365b95e62b47ef48f93a69ab734e623248a8a4079ecb/srsly-1.0.5.tar.gz (86kB) Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-build-vjnjnimo/srsly/setup.py", line 11, in <module> from Cython.Build import cythonize ModuleNotFoundError: No module named 'Cython' ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-vjnjnimo/srsly/

I give a docker route to reproduce to match the environment perfectly.

Entities

https://github.com/explosion/srsly/pull/44 https://github.com/explosion/srsly/blob/v1.0.5/setup.py#L11
opened by DomHudson 2
srsly.write_json segfaults trying to write numpy types
Hi,

I was using srsly for json serialization and it killed my jupyter kernels without producing any errors. It turns out it's due to ujson segmentation faults trying to serialize types it doesn't understand. It's quite easy to reproduce though:

import numpy as np import srsly f = np.float32() srsly.write_json("x.json", f)

Believe this has now been fixed in ujson as part of https://github.com/ultrajson/ultrajson/commit/53f85b1bd6e4f27a3e4cdc605518c48a6c7e7e9e (issue https://github.com/ultrajson/ultrajson/issues/294). Doing an equivalent test with the latest ujson (version 3.0) gives a sensible error.

Prior to that fix, it looked like some people were abandoning ujson (https://github.com/noirbizarre/flask-restplus/issues/589) and there was a Pandas fork (https://github.com/pandas-dev/pandas/tree/master/pandas/_libs/src/ujson/python) before it started being maintained again.

Are there any plans to update the version used by srsly?
opened by jjjamie 2

srsly.read_json is broken

code:

import srsly

print(srsly.read_json("foo.json"))

outputs:

Traceback (most recent call last):
  File "/tmp/foo.py", line 4, in <module>
    print(srsly.read_json("foo.json"))
  File "/Users/yohei_tamura/work/lilly/.venv/lib/python3.8/site-packages/srsly/_json_api.py", line 52, in read_json
    return ujson.load(f)
ValueError: Expected object or value

srsly version: 2.2.0 python version: 3.8.1

opened by tamuhey 2

Release v1.0.0 was removed from PyPI.org
The release package for 1.0.0 was removed from PyPI.org. Doing this causes problems for many automated build systems and pipelines. Removing packages from repos is frowned upon. Please replace the package in PyPI. It it is broken in some way, please release a v1.0.1 instead of removing 1.0.0.

pip3 install srsly==1.0.0 Collecting srsly==1.0.0 Could not find a version that satisfies the requirement srsly==1.0.0 (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.1.0, 0.2.0, 2.0.0.dev0, 2.0.0.dev1, 2.0.0) No matching distribution found for srsly==1.0.0
opened by robbwagoner 2

Releases(v2.4.5)

v2.4.5(Oct 18, 2022)
Update to cloudpickle v2.2.0 for Python 3.11 support.

Add Python 3.11 wheels.

Source code(tar.gz)
Source code(zip)
v1.0.6(Oct 18, 2022)
Drop support for Python 3.5 and earlier.

Update to cloudpickle v2.2.0 for Python 3.11 support.

Add Python 3.11 wheels.

Source code(tar.gz)
Source code(zip)
v2.4.4(Jul 20, 2022)
Port https://github.com/ultrajson/ultrajson/pull/550 and https://github.com/ultrajson/ultrajson/pull/555 to fix incorrect handling of invalid surrogate pair characters (CVE-2022-31116)

Source code(tar.gz)
Source code(zip)
v2.4.3(Apr 13, 2022)
Port https://github.com/ultrajson/ultrajson/pull/519 to fix buffer overflows (CVE-2021-4595)

Source code(tar.gz)
Source code(zip)
v2.4.1(Apr 12, 2021)
Use str with chdir in setup

Fix type of write_jsonl

Fix json_loads behaviour for string "-"

Source code(tar.gz)
Source code(zip)
v2.4.0(Feb 1, 2021)
Support custom msgpack serializers and deserializers via function registries (mostly internals).

Update setup.

Source code(tar.gz)
Source code(zip)
v1.0.3(Nov 4, 2020)
Update for python 3.9

Source code(tar.gz)
Source code(zip)
v2.3.1(Nov 4, 2020)
Update for python 3.9

Source code(tar.gz)
Source code(zip)
v2.3.0(Oct 1, 2020)
Adjust argument names.

Expose data loaders for spaCy v3.

Source code(tar.gz)
Source code(zip)
v2.2.0(Jun 30, 2020)
Remove unsafe YAML implementations, raising if called (see #28)

Source code(tar.gz)
Source code(zip)
v2.1.0(Jun 21, 2020)
Include vendorized ruamel.yaml.

Add YAML API for saving and loading YAML.

Source code(tar.gz)
Source code(zip)
v2.0.1(Mar 4, 2020)
Support cupy.ndarray in msgpack_numpy.

Make msgpack raise correct errors for unserializable objects if numpy is not installed.

Source code(tar.gz)
Source code(zip)
v1.0.2(Mar 4, 2020)
Support cupy.ndarray in msgpack_numpy.

Make msgpack raise correct errors for unserializable objects if numpy is not installed.

Source code(tar.gz)
Source code(zip)
v1.0.1(Jan 6, 2020)
Fix source distribution.

Source code(tar.gz)
Source code(zip)
v2.0.0(Jan 2, 2020)
Port over esnme/ultrajson#323 by @timdawborn.

Drop support for Python 2.7 and 3.5.

Tidy up code and remove compatibility helpers.

Add type hints.

Improve setup and compilation.

Source code(tar.gz)
Source code(zip)
v1.0.0(Jan 2, 2020)
Add official support for Python 3.8.

Fix typo in README.md.

Thanks to @pfvosi for the pull request!
Source code(tar.gz)
Source code(zip)
v0.2.0(Oct 28, 2019)
Fix issue #10: Update cloudpickle to 1.2.2 to support Python 3.8.

Fix issue #13: Add append_new_line option to write_jsonl to control whether newline is appended.

Thanks to @nyejon for the pull request!
Source code(tar.gz)
Source code(zip)
v0.1.0(Aug 18, 2019)
Add read_gzip_json and write_gzip_json for working with compressed JSON.

Thanks to @polm for the pull request!
Source code(tar.gz)
Source code(zip)
v0.0.7(Jun 13, 2019)
Fix issue #6: Don't write to JSONL as "a" (append) by default.

Source code(tar.gz)
Source code(zip)
v0.0.6(Jun 7, 2019)
Fix issue #4: Fix memory leak.

Thanks to @sadovnychyi for the pull request and to @Adriandorr for the fix.
Source code(tar.gz)
Source code(zip)
v0.0.5(Dec 2, 2018)
Fix length limits on msgpack serialization.

Source code(tar.gz)
Source code(zip)
v0.0.4(Dec 2, 2018)
Move ujson to srsly.ujson for API consistency.

Add pickle utilities: pickle_loads and pickle_dumps.

Add msgpack utilities: msgpack_loads, msgpack_dumps, write_msgpack, read_msgpack.

Improve test suite.

Source code(tar.gz)
Source code(zip)
v0.0.3(Dec 2, 2018)
Expose json_dumps and json_loads helpers.

Add support for sort_keys via json.

Source code(tar.gz)
Source code(zip)
v0.0.2(Dec 2, 2018)
Fix tests across Python versions and platforms.

Improve file path handling.

Source code(tar.gz)
Source code(zip)
v0.0.1(Dec 1, 2018)

Source code(tar.gz)
Source code(zip)

Owner

Explosion

A software company specializing in developer tools for Artificial Intelligence and Natural Language Processing

GitHub

FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

19.6k Jan 1, 2023

Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

jsonpickle jsonpickle is a library for the two-way conversion of complex Python objects and JSON. jsonpickle builds upon the existing JSON encoders, s

1.1k Jan 2, 2023

🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

Related tags

Overview

srsly: Modern high-performance serialization utilities for Python

Motivation

Installation

API

JSON

function srsly.json_dumps

function srsly.json_loads

function srsly.write_json

function srsly.read_json

function srsly.write_gzip_json

function srsly.read_gzip_json

function srsly.write_jsonl

function srsly.read_jsonl

function srsly.is_json_serializable

msgpack

function srsly.msgpack_dumps

function srsly.msgpack_loads

function srsly.write_msgpack

function srsly.read_msgpack

pickle

function srsly.pickle_dumps

function srsly.pickle_loads

YAML

function srsly.yaml_dumps

function srsly.yaml_loads

function srsly.write_yaml

function srsly.read_yaml

function srsly.is_yaml_serializable

Comments

Description

Types of change

Checklist

Summary

Reproduce

Entities

Releases(v2.4.5)

v2.4.5(Oct 18, 2022)

v1.0.6(Oct 18, 2022)

v2.4.4(Jul 20, 2022)

v2.4.3(Apr 13, 2022)

v2.4.1(Apr 12, 2021)

v2.4.0(Feb 1, 2021)

v1.0.3(Nov 4, 2020)

v2.3.1(Nov 4, 2020)

v2.3.0(Oct 1, 2020)

v2.2.0(Jun 30, 2020)

v2.1.0(Jun 21, 2020)

v2.0.1(Mar 4, 2020)

v1.0.2(Mar 4, 2020)

v1.0.1(Jan 6, 2020)

v2.0.0(Jan 2, 2020)

v1.0.0(Jan 2, 2020)

v0.2.0(Oct 28, 2019)

v0.1.0(Aug 18, 2019)

v0.0.7(Jun 13, 2019)

v0.0.6(Jun 7, 2019)

v0.0.5(Dec 2, 2018)

v0.0.4(Dec 2, 2018)

v0.0.3(Dec 2, 2018)

v0.0.2(Dec 2, 2018)

v0.0.1(Dec 1, 2018)

Owner

Explosion

FlatBuffers: Memory Efficient Serialization Library

Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

Ultra fast JSON decoder and encoder written in C with Python bindings

simplejson is a simple, fast, extensible JSON encoder/decoder for Python

Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

Crappy tool to convert .scw files to .json and and vice versa.

A lightweight library for converting complex objects to and from simple Python datatypes.

Extended pickling support for Python objects

Generic ASN.1 library for Python

serialize all of python

Python wrapper around rapidjson

Python bindings for the simdjson project.

Simple, light-weight config handling through python data classes with to/from JSON serialization/deserialization.

Console to handle object storage using JSON serialization and deserealization.

`function` `srsly.json_dumps`

`function` `srsly.json_loads`

`function` `srsly.write_json`

`function` `srsly.read_json`

`function` `srsly.write_gzip_json`

`function` `srsly.read_gzip_json`

`function` `srsly.write_jsonl`

`function` `srsly.read_jsonl`

`function` `srsly.is_json_serializable`

`function` `srsly.msgpack_dumps`

`function` `srsly.msgpack_loads`

`function` `srsly.write_msgpack`

`function` `srsly.read_msgpack`

`function` `srsly.pickle_dumps`

`function` `srsly.pickle_loads`

`function` `srsly.yaml_dumps`

`function` `srsly.yaml_loads`

`function` `srsly.write_yaml`

`function` `srsly.read_yaml`

`function` `srsly.is_yaml_serializable`