simplejson is a simple, fast, extensible JSON encoder/decoder for Python



simplejson is a simple, fast, complete, correct and extensible JSON <> encoder and decoder for Python 3.3+ with legacy support for Python 2.5+. It is pure Python code with no dependencies, but includes an optional C extension for a serious speed boost.

The latest documentation for simplejson can be read online here:

simplejson is the externally maintained development version of the json library included with Python (since 2.6). This version is tested with the latest Python 3.8 and maintains backwards compatibility with Python 3.3+ and the legacy Python 2.5 - Python 2.7 releases.

The encoder can be specialized to provide serialization in any kind of situation, without any special support by the objects to be serialized (somewhat like pickle). This is best done with the default kwarg to dumps.

The decoder can handle incoming JSON strings of any specified encoding (UTF-8 by default). It can also be specialized to post-process JSON objects with the object_hook or object_pairs_hook kwargs. This is particularly useful for implementing protocols such as JSON-RPC that have a richer type system than JSON itself.

For those of you that have legacy systems to maintain, there is a very old fork of simplejson in the python2.2 branch that supports Python 2.2. This is based on a very old version of simplejson, is not maintained, and should only be used as a last resort.

  • pip wheel segfaults after simplejson is installed

    pip wheel segfaults after simplejson is installed

    This only happens after simplejson is installed so I'm logging the issue here.

    The backtrace is a bit skimpy, but unfortunately this doesn't reproduce with python-dbg.

    Neither it does reproduce if I compile a custom 2.7.9 Python. Everyone should be using that, but alas, Ubuntu's LTS release comes with 2.7.3, unlikely many people will have 2.7.9 :cry:

    [email protected]:~$ pip --version
    pip 6.0.6 from /usr/local/lib/python2.7/dist-packages (python 2.7)
    [email protected]:~$ easy_install --version
    setuptools 11.3.1
    [email protected]:~$ python --version
    Python 2.7.3
    [email protected]:~$ lsb
    lsblk        lsb_release
    [email protected]:~$ lsb_release
    No LSB modules are available.
    [email protected]:~$ lsb_release -a
    No LSB modules are available.
    Distributor ID: Ubuntu
    Description:    Ubuntu 12.04.5 LTS
    Release:        12.04
    Codename:       precise
    [email protected]:~$ pip wheel --wheel-dir=. simplejson
    Collecting simplejson
      Using cached simplejson-3.6.5.tar.gz
    Building wheels for collected packages: simplejson
      Running bdist_wheel for simplejson
      Destination directory: /home/ionel
    Successfully built simplejson
    [email protected]:~$ rm -rf .ve
    virtualenv .ve --system-site-packages
    New python executable in .ve/bin/python
    Installing setuptools, pip...done.
    [email protected]:~$ .ve/bin/pip install -I simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Processing ./simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Installing collected packages: simplejson
    Successfully installed simplejson-3.6.5
    [email protected]:~$ .ve/bin/pip install -I simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Processing ./simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Installing collected packages: simplejson
    Successfully installed simplejson-3.6.5
    Segmentation fault (core dumped)
    [email protected]:~$ LD_PRELOAD=/lib/x86_64-linux-gnu/ .ve/bin/pip install -I simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Processing ./simplejson-3.6.5-cp27-none-linux_x86_64.whl
    Installing collected packages: simplejson
    Successfully installed simplejson-3.6.5
    *** Segmentation fault
    Register dump:
     RAX: 0000000000000000   RBX: 00007f57897ee4e0   RCX: 0000000001ed4f00
     RDX: 0000000001ed4f18   RSI: 0000000000000000   RDI: 00007f57897ee4e0
     RBP: 0000000000555de0   R8 : 00000000024c2190   R9 : 0000000002486840
     R10: 0000000000000008   R11: 0000000000000206   R12: 000000000000000f
     R13: 00007f57897ee4e0   R14: 0000000000000000   R15: 0000000000000002
     RSP: 00007fffaf78abd0
     RIP: 0000000000555de8   EFLAGS: 00010206
     CS: 0033   FS: 0000   GS: 0000
     Trap: 0000000e   Error: 00000004   OldMask: 00000000   CR2: 000000a9
     FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
     RIP: 00000000   RDP: 00000000
     ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
     ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
     ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
     ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
     mxcsr: 1fa0
     XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
     XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
     XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
     XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
     XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
     XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
     XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
     XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
    Memory map:
    00400000-00672000 r-xp 00000000 08:01 6829994                            /home/ionel/.ve/bin/python
    00871000-00872000 r--p 00271000 08:01 6829994                            /home/ionel/.ve/bin/python
    00872000-008db000 rw-p 00272000 08:01 6829994                            /home/ionel/.ve/bin/python
    008db000-008ed000 rw-p 00000000 00:00 0
    01703000-02643000 rw-p 00000000 00:00 0                                  [heap]
    7f578897b000-7f5788988000 r-xp 00000000 08:01 6043731                    /usr/lib/python2.7/lib-dynload/
    7f5788988000-7f5788b87000 ---p 0000d000 08:01 6043731                    /usr/lib/python2.7/lib-dynload/
    7f5788b87000-7f5788b88000 r--p 0000c000 08:01 6043731                    /usr/lib/python2.7/lib-dynload/
    7f5788b88000-7f5788b89000 rw-p 0000d000 08:01 6043731                    /usr/lib/python2.7/lib-dynload/
    7f5788b89000-7f5788b98000 r-xp 00000000 08:01 524293                     /lib/x86_64-linux-gnu/
    7f5788b98000-7f5788d97000 ---p 0000f000 08:01 524293                     /lib/x86_64-linux-gnu/
    7f5788d97000-7f5788d98000 r--p 0000e000 08:01 524293                     /lib/x86_64-linux-gnu/
    7f5788d98000-7f5788d99000 rw-p 0000f000 08:01 524293                     /lib/x86_64-linux-gnu/
    7f5788d99000-7f5788da0000 r-xp 00000000 08:01 6043736                    /usr/lib/python2.7/lib-dynload/
    7f5788da0000-7f5788f9f000 ---p 00007000 08:01 6043736                    /usr/lib/python2.7/lib-dynload/
    7f5788f9f000-7f5788fa0000 r--p 00006000 08:01 6043736                    /usr/lib/python2.7/lib-dynload/
    7f5788fa0000-7f5788fa2000 rw-p 00007000 08:01 6043736                    /usr/lib/python2.7/lib-dynload/
    7f5788fa2000-7f5788fa8000 r-xp 00000000 08:01 6043759                    /usr/lib/python2.7/lib-dynload/
    7f5788fa8000-7f57891a7000 ---p 00006000 08:01 6043759                    /usr/lib/python2.7/lib-dynload/
    7f57891a7000-7f57891a8000 r--p 00005000 08:01 6043759                    /usr/lib/python2.7/lib-dynload/
    7f57891a8000-7f57891aa000 rw-p 00006000 08:01 6043759                    /usr/lib/python2.7/lib-dynload/
    7f57891aa000-7f57891d1000 r-xp 00000000 08:01 524520                     /lib/x86_64-linux-gnu/
    7f57891d1000-7f57893d1000 ---p 00027000 08:01 524520                     /lib/x86_64-linux-gnu/
    7f57893d1000-7f57893d3000 r--p 00027000 08:01 524520                     /lib/x86_64-linux-gnu/
    7f57893d3000-7f57893d4000 rw-p 00029000 08:01 524520                     /lib/x86_64-linux-gnu/
    7f57893d4000-7f57893e2000 r-xp 00000000 08:01 6043761                    /usr/lib/python2.7/lib-dynload/
    7f57893e2000-7f57895e1000 ---p 0000e000 08:01 6043761                    /usr/lib/python2.7/lib-dynload/
    7f57895e1000-7f57895e2000 r--p 0000d000 08:01 6043761                    /usr/lib/python2.7/lib-dynload/
    7f57895e2000-7f57895e4000 rw-p 0000e000 08:01 6043761                    /usr/lib/python2.7/lib-dynload/
    7f57895e4000-7f57895ee000 r-xp 00000000 08:01 6830780                    /home/ionel/.ve/lib/python2.7/site-packages/simplejson/
    7f57895ee000-7f57897ed000 ---p 0000a000 08:01 6830780                    /home/ionel/.ve/lib/python2.7/site-packages/simplejson/
    7f57897ed000-7f57897ee000 r--p 00009000 08:01 6830780                    /home/ionel/.ve/lib/python2.7/site-packages/simplejson/
    7f57897ee000-7f57897ef000 rw-p 0000a000 08:01 6830780                    /home/ionel/.ve/lib/python2.7/site-packages/simplejson/
    7f57897ef000-7f57897f3000 r-xp 00000000 08:01 559770                     /lib/x86_64-linux-gnu/
    7f57897f3000-7f57899f2000 ---p 00004000 08:01 559770                     /lib/x86_64-linux-gnu/
    7f57899f2000-7f57899f3000 r--p 00003000 08:01 559770                     /lib/x86_64-linux-gnu/
    7f57899f3000-7f57899f4000 rw-p 00004000 08:01 559770                     /lib/x86_64-linux-gnu/
    7f57899f4000-7f57899ff000 r-xp 00000000 08:01 6043756                    /usr/lib/python2.7/lib-dynload/
    7f57899ff000-7f5789bfe000 ---p 0000b000 08:01 6043756                    /usr/lib/python2.7/lib-dynload/
    7f5789bfe000-7f5789bff000 r--p 0000a000 08:01 6043756                    /usr/lib/python2.7/lib-dynload/
    7f5789bff000-7f5789c00000 rw-p 0000b000 08:01 6043756                    /usr/lib/python2.7/lib-dynload/
    7f5789c00000-7f5789c04000 r-xp 00000000 08:01 6043742                    /usr/lib/python2.7/lib-dynload/
    7f5789c04000-7f5789e03000 ---p 00004000 08:01 6043742                    /usr/lib/python2.7/lib-dynload/
    7f5789e03000-7f5789e04000 r--p 00003000 08:01 6043742                    /usr/lib/python2.7/lib-dynload/
    7f5789e04000-7f5789e06000 rw-p 00004000 08:01 6043742                    /usr/lib/python2.7/lib-dynload/
    7f5789e06000-7f578a0cf000 r--p 00000000 08:01 6035410                    /usr/lib/locale/locale-archive
    7f578a0cf000-7f578a0e2000 r-xp 00000000 08:01 6043735                    /usr/lib/python2.7/lib-dynload/
    7f578a0e2000-7f578a2e1000 ---p 00013000 08:01 6043735                    /usr/lib/python2.7/lib-dynload/
    7f578a2e1000-7f578a2e2000 r--p 00012000 08:01 6043735                    /usr/lib/python2.7/lib-dynload/
    7f578a2e2000-7f578a2e3000 rw-p 00013000 08:01 6043735                    /usr/lib/python2.7/lib-dynload/
    7f578a2e3000-7f578a2f7000 r-xp 00000000 08:01 6043764                    /usr/lib/python2.7/lib-dynload/
    7f578a2f7000-7f578a4f6000 ---p 00014000 08:01 6043764                    /usr/lib/python2.7/lib-dynload/
    7f578a4f6000-7f578a4f7000 r--p 00013000 08:01 6043764                    /usr/lib/python2.7/lib-dynload/
    7f578a4f7000-7f578a4fb000 rw-p 00014000 08:01 6043764                    /usr/lib/python2.7/lib-dynload/
    7f578a4fb000-7f578a51a000 r-xp 00000000 08:01 6043753                    /usr/lib/python2.7/lib-dynload/
    7f578a51a000-7f578a719000 ---p 0001f000 08:01 6043753                    /usr/lib/python2.7/lib-dynload/
    7f578a719000-7f578a71a000 r--p 0001e000 08:01 6043753                    /usr/lib/python2.7/lib-dynload/
    7f578a71a000-7f578a723000 rw-p 0001f000 08:01 6043753                    /usr/lib/python2.7/lib-dynload/
    7f578a723000-7f578a764000 rw-p 00000000 00:00 0
    7f578a764000-7f578a785000 r-xp 00000000 08:01 6043762                    /usr/lib/python2.7/lib-dynload/
    7f578a785000-7f578a984000 ---p 00021000 08:01 6043762                    /usr/lib/python2.7/lib-dynload/
    7f578a984000-7f578a985000 r--p 00020000 08:01 6043762                    /usr/lib/python2.7/lib-dynload/
    7f578a985000-7f578a989000 rw-p 00021000 08:01 6043762                    /usr/lib/python2.7/lib-dynload/
    7f578a989000-7f578ab90000 rw-p 00000000 00:00 0
    7f578ab90000-7f578ab93000 r-xp 00000000 08:01 6043740                    /usr/lib/python2.7/lib-dynload/
    7f578ab93000-7f578ad92000 ---p 00003000 08:01 6043740                    /usr/lib/python2.7/lib-dynload/
    7f578ad92000-7f578ad93000 r--p 00002000 08:01 6043740                    /usr/lib/python2.7/lib-dynload/
    7f578ad93000-7f578ad95000 rw-p 00003000 08:01 6043740                    /usr/lib/python2.7/lib-dynload/
    7f578ad95000-7f578adaa000 r-xp 00000000 08:01 524332                     /lib/x86_64-linux-gnu/
    7f578adaa000-7f578afa9000 ---p 00015000 08:01 524332                     /lib/x86_64-linux-gnu/
    7f578afa9000-7f578afaa000 r--p 00014000 08:01 524332                     /lib/x86_64-linux-gnu/
    7f578afaa000-7f578afab000 rw-p 00015000 08:01 524332                     /lib/x86_64-linux-gnu/
    7f578afab000-7f578b160000 r-xp 00000000 08:01 572001                     /lib/x86_64-linux-gnu/
    7f578b160000-7f578b35f000 ---p 001b5000 08:01 572001                     /lib/x86_64-linux-gnu/
    7f578b35f000-7f578b363000 r--p 001b4000 08:01 572001                     /lib/x86_64-linux-gnu/
    7f578b363000-7f578b365000 rw-p 001b8000 08:01 572001                     /lib/x86_64-linux-gnu/
    7f578b365000-7f578b36a000 rw-p 00000000 00:00 0
    7f578b36a000-7f578b465000 r-xp 00000000 08:01 572018                     /lib/x86_64-linux-gnu/
    7f578b465000-7f578b664000 ---p 000fb000 08:01 572018                     /lib/x86_64-linux-gnu/
    7f578b664000-7f578b665000 r--p 000fa000 08:01 572018                     /lib/x86_64-linux-gnu/
    7f578b665000-7f578b666000 rw-p 000fb000 08:01 572018                     /lib/x86_64-linux-gnu/
    7f578b666000-7f578b67c000 r-xp 00000000 08:01 524517                     /lib/x86_64-linux-gnu/
    7f578b67c000-7f578b87b000 ---p 00016000 08:01 524517                     /lib/x86_64-linux-gnu/
    7f578b87b000-7f578b87c000 r--p 00015000 08:01 524517                     /lib/x86_64-linux-gnu/
    7f578b87c000-7f578b87d000 rw-p 00016000 08:01 524517                     /lib/x86_64-linux-gnu/
    7f578b87d000-7f578ba2f000 r-xp 00000000 08:01 527949                     /lib/x86_64-linux-gnu/
    7f578ba2f000-7f578bc2e000 ---p 001b2000 08:01 527949                     /lib/x86_64-linux-gnu/
    7f578bc2e000-7f578bc49000 r--p 001b1000 08:01 527949                     /lib/x86_64-linux-gnu/
    7f578bc49000-7f578bc54000 rw-p 001cc000 08:01 527949                     /lib/x86_64-linux-gnu/
    7f578bc54000-7f578bc58000 rw-p 00000000 00:00 0
    7f578bc58000-7f578bcac000 r-xp 00000000 08:01 527944                     /lib/x86_64-linux-gnu/
    7f578bcac000-7f578beac000 ---p 00054000 08:01 527944                     /lib/x86_64-linux-gnu/
    7f578beac000-7f578beaf000 r--p 00054000 08:01 527944                     /lib/x86_64-linux-gnu/
    7f578beaf000-7f578beb6000 rw-p 00057000 08:01 527944                     /lib/x86_64-linux-gnu/
    7f578beb6000-7f578beb8000 r-xp 00000000 08:01 572024                     /lib/x86_64-linux-gnu/
    7f578beb8000-7f578c0b7000 ---p 00002000 08:01 572024                     /lib/x86_64-linux-gnu/
    7f578c0b7000-7f578c0b8000 r--p 00001000 08:01 572024                     /lib/x86_64-linux-gnu/
    7f578c0b8000-7f578c0b9000 rw-p 00002000 08:01 572024                     /lib/x86_64-linux-gnu/
    7f578c0b9000-7f578c0bb000 r-xp 00000000 08:01 572029                     /lib/x86_64-linux-gnu/
    7f578c0bb000-7f578c2bb000 ---p 00002000 08:01 572029                     /lib/x86_64-linux-gnu/
    7f578c2bb000-7f578c2bc000 r--p 00002000 08:01 572029                     /lib/x86_64-linux-gnu/
    7f578c2bc000-7f578c2bd000 rw-p 00003000 08:01 572029                     /lib/x86_64-linux-gnu/
    7f578c2bd000-7f578c2d5000 r-xp 00000000 08:01 571994                     /lib/x86_64-linux-gnu/
    7f578c2d5000-7f578c4d4000 ---p 00018000 08:01 571994                     /lib/x86_64-linux-gnu/
    7f578c4d4000-7f578c4d5000 r--p 00017000 08:01 571994                     /lib/x86_64-linux-gnu/
    7f578c4d5000-7f578c4d6000 rw-p 00018000 08:01 571994                     /lib/x86_64-linux-gnu/
    7f578c4d6000-7f578c4da000 rw-p 00000000 00:00 0
    7f578c4da000-7f578c4de000 r-xp 00000000 08:01 572025                     /lib/x86_64-linux-gnu/
    7f578c4de000-7f578c6dd000 ---p 00004000 08:01 572025                     /lib/x86_64-linux-gnu/
    7f578c6dd000-7f578c6de000 r--p 00003000 08:01 572025                     /lib/x86_64-linux-gnu/
    7f578c6de000-7f578c6df000 rw-p 00004000 08:01 572025                     /lib/x86_64-linux-gnu/
    7f578c6df000-7f578c701000 r-xp 00000000 08:01 571993                     /lib/x86_64-linux-gnu/
    7f578c734000-7f578c838000 rw-p 00000000 00:00 0
    7f578c869000-7f578c8f1000 rw-p 00000000 00:00 0
    7f578c8fd000-7f578c8fe000 rw-p 00000000 00:00 0
    7f578c8fe000-7f578c8ff000 rwxp 00000000 00:00 0
    7f578c8ff000-7f578c901000 rw-p 00000000 00:00 0
    7f578c901000-7f578c902000 r--p 00022000 08:01 571993                     /lib/x86_64-linux-gnu/
    7f578c902000-7f578c904000 rw-p 00023000 08:01 571993                     /lib/x86_64-linux-gnu/
    7fffaf757000-7fffaf78c000 rw-p 00000000 00:00 0                          [stack]
    7fffaf7fe000-7fffaf800000 r-xp 00000000 00:00 0                          [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
    Segmentation fault (core dumped)
    opened by ionelmc 27
  • Add support for `datetime`

    Add support for `datetime`

    Should simplejson support datetime objects? If so then what is the natural encoding?

    opened by mrocklin 23
  • simplejson.scanner.JSONDecodeError: Unpaired high surrogate

    simplejson.scanner.JSONDecodeError: Unpaired high surrogate


    Sorry for taking a while to report this. It took me some time to find the JSON (in about 2.5TB of data) and then I realised I should probably trim it down to the relevant section.

    I am unable to parse the following JSON with simplejson 3.1.0 under Linux. This JSON is taken from a result set from ElasticSearch and obviously trimmed down to be as small as possible.

    {"a": "\ud8e9"}

    I done some testing on a fresh vagrant Ubuntu Precise 64bit virtual machine but previously we have been seeing the error with lucid on EC2.

    The error get is;

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 398, in load
        use_decimal=use_decimal, **kw)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 454, in loads
        return _default_decoder.decode(s)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 374, in decode
        obj, end = self.raw_decode(s)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 393, in raw_decode
        return self.scan_once(s, idx=_w(s, idx).end())
    simplejson.scanner.JSONDecodeError: Unpaired high surrogate: line 1 column 8 (char 7)

    If I disable the speedups with simplejson._toggle_speedups(False) I get a (possibly) slightly more helpful error.

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 398, in load
        use_decimal=use_decimal, **kw)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 454, in loads
        return _default_decoder.decode(s)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 374, in decode
        obj, end = self.raw_decode(s)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 393, in raw_decode
        return self.scan_once(s, idx=_w(s, idx).end())
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 119, in scan_once
        return _scan_once(string, idx)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 90, in _scan_once
        _scan_once, object_hook, object_pairs_hook, memo)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 198, in JSONObject
        value, end = scan_once(s, end)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 87, in _scan_once
        return parse_string(string, idx + 1, encoding, strict)
      File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/", line 118, in py_scanstring
        raise JSONDecodeError(msg, s, end)
    simplejson.scanner.JSONDecodeError: Unpaired high surrogate: line 1 column 8 (char 7)

    However, I'm not totally sure where the blame lies here. I've done a bit of research and this is what I've found.

    • I mentioned before I only have this problem under Linux. Locally on my mac it works fine. I've not been able to test Windows. I was pointed to this issue by @bigkevmcd: which seems relevant but not conclusive.
    • I have tested this file with a few other languages for comparison - works fine in Java and Ruby. JavaScript seems to mostly work, although I noticed some display issues in the Chrome console.
    • I have discovered some other JSON documents cause the same error however they do work when I turn off the speedups. I've not been logging these as I'm turning off the speedups and trying again and then only logging those that still fail. I'll do another run (probably not until Monday now) and add logging of these so I can provide them.
    • In my tweet I was confused why it worked, failed and then worked after a pip re-install. This was with a different document and because simplejson was originally installed on the server with a package. Then when I pip installed it failed to install speedups - thus I "disabled" them without realising.

    So, yeah, that's what I've found. I'm not sure how helpful it is. It does seem that the JSON that work without c speedups (but not with) only are causing a bug but it also seems to me that unpaired surrogate issues should either always work or always not work...

    opened by d0ugal 19
  • item_sort_key always uses itemgetter in 3.11

    item_sort_key always uses itemgetter in 3.11

    Got a strange bug when upgrading 3.10.0 to 3.11.0 via pip.

    A test started failing that used a custom sort function via item_sort_key. It seems that setting this kwarg has the effect of enabling sorting using the default itemgetter in 3.11.0.

    This occurs when using the C encoder implementation.

    Python is 2.7.11 on Ubuntu 16.04.

    Using this code:

    import simplejson
    def sortit(x):
      return x[1]
    a = {
      "c": "3",
      "b": "1",
      "a": "2"
    print simplejson.dumps(a, item_sort_key=sortit)

    On 3.10.0:

    $ python
    {"b": "1", "c": "2", "a": "3"}

    And on 3.11.0:

    $ python
    {"a": "2", "b": "1", "c": "3"}
    opened by doozr 18
  • JSON encoding helper hook

    JSON encoding helper hook

    I was wondering if you'd be open to including a helper hook for encoding arbitrary JSON objects.

    My use case is that I have classes which subclass dict, but need to have mutated keys before serialization to JSON. Because they subclass dict, they don't get picked up by the default() method.

    I'd like to see something where an optional hook can be called to return a more serializable representation (which may need recursion to serialize nested objects). For example:

    class Example(object):
        def for_json(self):
            return {'example': 1}

    This would be very similar to how the _asdict hook is called currently, but would use encode_listencode_obj (and _iterencode in Python), instead of the dict varieties of those functions.

    Any thoughts on this? Would you be open to including this feature into the simplejson library?

    I'm kind-of jumping the gun on this, because I haven't forked and implemented this, but I'm about to, so I wanted to ask early enough that I might not need to create a forked version on PyPI. :)

    opened by shakefu 16
  • use __json__ attribute to encode custom objects

    use __json__ attribute to encode custom objects

    I'm currently working on an issue where I'm writing a library that will interface with other moving parts that use simplejson internally. However, I need to be able to pass them a custom object and have that be serialized. Unfortunately, since the json encoding is done within these projects and isn't directly called by my code, I can't pass it a custom encoder.

    Pyramid (one of the moving parts I'm using) has a useful function that will use the json attribute to encode a custom object, but the other one I'm using does not have that. While searching around, I found this patch from the repo.

    It would be awesome if this behavior could be included in simplejson by default. The current behavior of using the output from repr(o) makes it very difficult to have my object serializable by simplejson in an agnostic manner, since I'm not working with the library directly.

    opened by chason 13
  • RFC: Use singledispatch for registering additional custom serializer functions

    RFC: Use singledispatch for registering additional custom serializer functions

    Using json.dumps() gets slightly complicated when you try to serialize more complex objects, like datetime.datetime or uuid.UUID objects, as simplejson and the built-in json module do not know how to serialize these objects:

    import simplejson as json
    import uuid
    from datetime import datetime
    # Traceback (most recent call last):
    #   File "<stdin>", line 1, in <module>
    #   File ".../simplejson/", line 382, in dumps
    #     return _default_encoder.encode(obj)
    #   File ".../simplejson/", line 385, in encode
    #     chunks = list(chunks)
    #   File ".../simplejson/", line 770, in _iterencode
    #     for chunk in _iterencode_dict(o, _current_indent_level):
    #   File ".../simplejson/", line 727, in _iterencode_dict
    #     for chunk in chunks:
    #   File ".../simplejson/", line 790, in _iterencode
    #     o = _default(o)
    #   File ".../simplejson/", line 88, in _method
    #     return method.__get__(instance, owner)(*args, **kwargs)
    #   File ".../simplejson/", line 360, in default
    #     o.__class__.__name__)
    # TypeError: Object of type datetime is not JSON serializable
    json.dumps({'uuid': uuid.uuid4()})
    # Traceback (most recent call last):
    #   File "<stdin>", line 1, in <module>
    #   File ".../simplejson/", line 382, in dumps
    #     return _default_encoder.encode(obj)
    #   File ".../simplejson/", line 385, in encode
    #     chunks = list(chunks)
    #   File ".../simplejson/", line 770, in _iterencode
    #     for chunk in _iterencode_dict(o, _current_indent_level):
    #   File ".../simplejson/", line 727, in _iterencode_dict
    #     for chunk in chunks:
    #   File ".../simplejson/", line 790, in _iterencode
    #     o = _default(o)
    #   File ".../simplejson/", line 88, in _method
    #     return method.__get__(instance, owner)(*args, **kwargs)
    #   File ".../simplejson/", line 360, in default
    #     o.__class__.__name__)
    # TypeError: Object of type UUID is not JSON serializable

    There are two workarounds for this:

    1. Subclass simplejson.encoder.JSONEncoder and override its default method, then explicitly call JSONEncoder.encode() everywhere (this is exactly what Django does).

    2. Define a function that accepts a single argument and serializes it appropriately, then call json.dumps() with the default argument everywhere:

      def my_encoder(obj):
          if isinstance(obj, datetime):
              return obj.isoformat()
          elif isinstance(obj, uuid.UUID):
              return str(obj)
              raise TypeError("Cannot serialize object of type '%s'" % (type(obj)))
      json.dumps({'now':, 'uuid': uuid.uuid4()}, default=my_encoder)
      # '{"now": "2018-01-21T12:34:42.993445", "uuid": "5f9a8bbe-d3b2-4a32-8bc0-83ac87cb195c"}'

    Both of these workarounds require users to modify/override the custom serializer function rather than extending it, and then to use the modified code everywhere instead of directly using the built-in json module or simplejson. This generally means frameworks like Django force all users to use their own serializers with their own serializer functions. It's also rather confusing for new Django users, who ask questions like this on Stack Overflow "How do I encode a UUID to make it JSON serializable?".

    Furthermore, for a non-trivial project, the custom default function can become unwieldy and begins to look exactly like the usecase for singledispatch.

    This PR simply uses a modified singledispatch (called singledispatchmethod) decorator to extend the default function for specific object types. This makes the simplejson.encoder.JSONEncoder.default method better adhere to the Open/Closed Principle - open to extension but closed to modification (or override). This decorator should also be completely backwards compatible - subclasses of JSONEncoder can still override it, and calls to simplejson.dumps() with a default= argument will still use the given custom encoder function.

    This change allows frameworks to define and register those functions anywhere that is automatically imported (like, for instance, django/

    import simplejson as json
    import uuid
    from datetime import datetime
    def jsonify_datetime(encoder, dt):
        return dt.isoformat()
    def jsonify_uuid(encoder, uuid_):
        return str(uuid_)

    That's it - now every user of Django can simply call simplejson.dumps() directly and it will be able to properly serialize datetimes and UUIDs:

    import simplejson as json
    import uuid
    from datetime import datetime
    json.dumps({'now':, 'uuid': uuid.uuid4()})
    # '{"now": "2018-01-22T03:00:49.847345", "uuid": "629a7f1e-3eb6-4bf3-a6cf-6717910e4a92"}'

    Now, I realize that Django is a huge framework, and it has DRF, which implements its own framework for registering serializers. But I think this PR has the potential to greatly simplify framework code.

    The singledispatch decorator does not work on instance methods, but there is a PR to fix this. I have included the two implementations (one naïve, one that is more complex but handles corner cases well) in this PR FOR REFERENCE ONLY. I will be removing them and replacing them the proper imports from functools once PR python/cpython#4987 or something like it is merged.

    This PR should also fix or obviate the needs to fix the following issues and PRs:

    • #149
    • #158
    • #159
    • #164

    I'm happy to respond to any criticisms or answer any questions. I would eventually like to push this API into Python's built-in json module, but it's written in C and I wanted to get feedback on this Python implementation before I do that.


    opened by blag 12
  • patch for running on Python 2.4

    patch for running on Python 2.4

    Here is a patch for running on Python 2.4. I know the docs state that it runs on Python 2.5 and later, but this is a very small change that is required for running Sphinx on RHEL5. Hopefully it's useful and will be merged. I'm new to git, so thought I would keep it simple by attaching a diff.

    Python 2.4 was raising a SyntaxError and the following change was made to fix this. I'm running simplejson 2.5 from PyPI along with Sphinx 1.1.3. I ran the unit tests with the change and they all pass.

    Cheers, Chris LeBlanc

    diff --git a/simplejson/ b/simplejson/ index 05dd05c..5710e67 100644 --- a/simplejson/ +++ b/simplejson/ @@ -394,10 +394,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr, elif value is False: yield buf + 'false' elif isinstance(value, (int, long)):

    •            yield ((buf + str(value))
    •                   if (not _bigint_as_string or
    •                       (-1 << 53) < value < (1 << 53))
    •                       else (buf + '"' + str(value) + '"'))
    •            if not _bigint_as_string or (-1 << 53) < value < (1 << 53):
    •                yield buf + str(value)
    •            else:
    •                yield buf + '"' + str(value) + '"'
           elif isinstance(value, float):
               yield buf + _floatstr(value)
           elif _use_decimal and isinstance(value, Decimal):
      @@ -487,10 +487,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr, elif value is False: yield 'false' elif isinstance(value, (int, long)):
    •            yield (str(value)
    •                   if (not _bigint_as_string or
    •                       (-1 << 53) < value < (1 << 53))
    •                       else ('"' + str(value) + '"'))
    •            if not _bigint_as_string or (-1 << 53) < value < (1 << 53):
    •                yield str(value)
    •            else:
    •                yield '"' + str(value) + '"'
           elif isinstance(value, float):
               yield _floatstr(value)
           elif _use_decimal and isinstance(value, Decimal):
      @@ -528,10 +528,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr, elif o is False: yield 'false' elif isinstance(o, (int, long)):
    •        yield (str(o)
    •               if (not _bigint_as_string or
    •                   (-1 << 53) < o < (1 << 53))
    •                   else ('"' + str(o) + '"'))
    •        if not _bigint_as_string or (-1 << 53) < o < (1 << 53):
    •            yield str(o)
    •        else:
    •            yield '"' + str(o) + '"'
       elif isinstance(o, float):
           yield _floatstr(o)
       elif isinstance(o, list):
    opened by crleblanc 11
  • Python 3.2 and Python 3.1 Compatibility

    Python 3.2 and Python 3.1 Compatibility

    In Python 3.2 and 3.1 you get a syntax error with something like this:

    x = u"Hello world"

    Only you have 3 lines incompatible with Python 3.1 and 3.2:

    opened by goinnn 11
  • Adding `int_as_string_bitcount` option

    Adding `int_as_string_bitcount` option

    • Generalizes bigint_as_string to allow other bit sized integers.
    opened by coldeasy 11
  • Add `RawJSON` to documentation

    Add `RawJSON` to documentation

    It appears that RawJSON is currently not documented.

    opened by jack1142 1
  •  Is it possible to delete the simplejson/tests directory when creating a binary package(wheel package)

    Is it possible to delete the simplejson/tests directory when creating a binary package(wheel package)

    OS: centos 7.9
    python: 3.8.2
    Package    Version
    ---------- -------
    pip        21.0
    setuptools 52.0.0
    wheel      0.36.2

    When i run command python bdist_wheel,i get the result as follows: image

    After modify one line

    #packages=['simplejson', 'simplejson.tests']

    i get the result as follows: image

    Size compare: image delete the simplejson/tests directory could reduce 28kb.

    So when running simplejson,whether it is necessary to package the simplejson/tests directory in the wheel package?

    opened by tklltkioc 2
  • OSS-Fuzz integration: initial integration of fuzzer

    OSS-Fuzz integration: initial integration of fuzzer


    Given the popularity of simplejson I was thinking that it would be nice to set up continuous fuzzing of simplejson, by way of OSS-Fuzz. In this PR: I have done exactly that, namely created the necessary logic from an OSS-Fuzz perspective to integrate simplejson. This includes developing initial fuzzers as well as integrating into OSS-Fuzz, however, it is preferable to have the fuzzers upstream so I included it in this PR - if you are happy with having the fuzzers here then I will remove them from the OSS-Fuzz repository.

    Essentially, OSS-Fuzz is a free service run by Google that performs continuous fuzzing of important open source projects. Python support was recently provided and it can also fuzz native extensions, i.e. for memory corruption errors. The only expectation of integrating into OSS-Fuzz is that bugs will be fixed. This is not a "hard" requirement in that no one enforces this and the main point is if bugs are not fixed then it is a waste of resources to run the fuzzers, which we would like to avoid.

    If you would like to integrate, the only thing I need is as list of email(s) that will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats. Notice the emails affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

    opened by DavidKorczynski 0
  • RecursionError on pathological input

    RecursionError on pathological input

    When passing the attached input to the following script, I get an unexpected RecursionError exception, while using the latest git version of simplejson.

    import sys
    import simplejson
    with open(sys.argv[1], 'rb') as f:
        data =
    except simplejson.errors.JSONDecodeError:
    except UnicodeDecodeError:
    $ python test.txt
    Traceback (most recent call last):
      File "", line 7, in <module>
      File "/home/user/ven/lib/python3.8/site-packages/simplejson/", line 525, in loads
        return _default_decoder.decode(s)
      File "/home/user/ven/lib/python3.8/site-packages/simplejson/", line 370, in decode
        obj, end = self.raw_decode(s)
      File "/home/user/ven/lib/python3.8/site-packages/simplejson/", line 400, in raw_decode
        return self.scan_once(s, idx=_w(s, idx).end())
    RecursionError: maximum recursion depth exceeded while decoding a JSON object from a unicode string


    opened by Google-Autofuzz 2
  • Serializing Iterables with custom types

    Serializing Iterables with custom types

    Long story short: I need to run dump(..., iterable_as_array=True) because of memory concerns. With this option, however, I lose control of how numpy.ndarray is serialized, for example. I do not need numpy.ndarray to be treated as a generic Iterable: I want, for example, to store the file name instead which I can perfectly do when iterable_as_array=False or I just use the built-in json library. iterable_as_array=True has an overwhelming priority which is hard to justify.

    opened by pulkin 0
  • AttributeError: 'Function' object has no attribute 'items'

    AttributeError: 'Function' object has no attribute 'items'

    from geoalchemy2 import WKBElement
    import simplejson as json
    json.dumps({'a': [WKBElement('abcde')]})

    I tried to use the default as parser to due with WKBElement, but the code is not going into the parser, how could I do the customized serialization?

    opened by DeoLeung 3
  • Feature Request: no newlines in base lists

    Feature Request: no newlines in base lists

    I have some content with long and sometimes deeply nested lists (from np.ndarray.tolist()). While it is readable to print with newlines for every element, it would IMO be much cleaner to omit newlines in the inner-most list, and replace with spaces. For example,


     "arr": [


     "arr": [
      [ 0.0, null ,0.468 ],
      [ 0.0, null, 0.688 ],
      [ 0.0, 0.32, null ],
      [ 0.0, 0.25, null ]

    Would this be considered? Or too much of a special case? Thanks,

    opened by jessexknight 2
  • modified: Allowed to specify alternative string for `NaN/Infinity/-Infinity` .

    modified: Allowed to specify alternative string for `NaN/Infinity/-Infinity` .

    cls and/or default can not replace specify string from nan/inf/-inf .

    this commit modified to allow to specify alternative string for NaN/Infinity/-Infinity .


    import simplejson as json
    print(json.dumps({'foo': 100}))
    print(json.dumps({'foo': 'str'}))
    print(json.dumps({'foo': None}))
    print(json.dumps({'foo': True}))
    print(json.dumps({'foo': False}))
    print(json.dumps({'foo': float('nan')}))
    print(json.dumps({'foo': float('inf')}))
    print(json.dumps({'foo': float('-inf')}))
        'foo': float('nan'),
        'bar': float('inf'),
        'baz': float('-inf'),
        'foo': float('nan'),
        'bar': float('inf'),
        'baz': float('-inf'),
    }, alternative_nan='nan'))
        'foo': float('nan'),
        'bar': float('inf'),
        'baz': float('-inf'),
    }, alternative_pos_inf='posinf'))
        'foo': float('nan'),
        'bar': float('inf'),
        'baz': float('-inf'),
    }, alternative_neg_inf='neginf'))
            'foo': float('nan'),
            'bar': float('inf'),
            'baz': float('-inf'),


    {"foo": 100}
    {"foo": "str"}
    {"foo": null}
    {"foo": true}
    {"foo": false}
    {"foo": NaN}
    {"foo": Infinity}
    {"foo": -Infinity}
    {"foo": NaN, "bar": Infinity, "baz": -Infinity}
    {"foo": nan, "bar": Infinity, "baz": -Infinity}
    {"foo": NaN, "bar": posinf, "baz": -Infinity}
    {"foo": NaN, "bar": Infinity, "baz": neginf}
    {"foo": nan, "bar": posinf, "baz": neginf}
    opened by youkinjoh 1
  • `ensure_ascii=False` could do with a C implementation

    `ensure_ascii=False` could do with a C implementation

    It's a real shame that there's no C implementation of encode_basestring - so the ensure_ascii=False codepath is so slow.

    I don't really have enough context to fully understand it, but it looks to me like encode_basestring ought to be simpler to implement in C than py_encode_basestring_ascii, which of course already has a C implementation.

    opened by richvdh 2
  • Provide manylinux1 wheel distribution on PyPI

    Provide manylinux1 wheel distribution on PyPI


    This package is already available in wheel form for OS X/Windows, however it would be great to add Linux to that list via the new manylinux1 variant:

    Many thanks :-)

    opened by edmorley 4
Python bindings for the simdjson project.

pysimdjson Python bindings for the simdjson project, a SIMD-accelerated JSON parser. If SIMD instructions are unavailable a fallback parser is used, m

Tyler Kennedy 474 Sep 24, 2021
Ultra fast JSON decoder and encoder written in C with Python bindings

UltraJSON UltraJSON is an ultra fast JSON encoder and decoder written in pure C with bindings for Python 3.6+. Install with pip: $ python -m pip insta

null 3.4k Sep 24, 2021
🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

srsly: Modern high-performance serialization utilities for Python This package bundles some of the best Python serialization libraries into one standa

Explosion 230 Sep 9, 2021
Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

jsonpickle jsonpickle is a library for the two-way conversion of complex Python objects and JSON. jsonpickle builds upon the existing JSON encoders, s

null 962 Sep 16, 2021
Generic ASN.1 library for Python

ASN.1 library for Python This is a free and open source implementation of ASN.1 types and codecs as a Python package. It has been first written to sup

Ilya Etingof 187 Sep 20, 2021
Crappy tool to convert .scw files to .json and and vice versa.

SCW-JSON-TOOL Crappy tool to convert .scw files to .json and vice versa. How to use Run file with two arguments: python <scw2json or j

Fred31 5 May 14, 2021
Python wrapper around rapidjson

python-rapidjson Python wrapper around RapidJSON Authors: Ken Robbins <[email protected]> Lele Gaifax <[email protected]> License: MIT License Sta

null 429 Sep 17, 2021
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. Overview Protocol Buffe

Protocol Buffers 50.8k Sep 23, 2021
MessagePack serializer implementation for Python[Python]

MessagePack for Python What's this MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JS

MessagePack 1.5k Sep 22, 2021
A lightweight library for converting complex objects to and from simple Python datatypes.

marshmallow: simplified object serialization marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, t

marshmallow-code 5.7k Sep 24, 2021
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Google 16.8k Sep 23, 2021
serialize all of python

dill serialize all of python About Dill dill extends python's pickle module for serializing and de-serializing python objects to the majority of the b

The UQ Foundation 1.5k Sep 23, 2021
Extended pickling support for Python objects

cloudpickle cloudpickle makes it possible to serialize Python constructs not supported by the default pickle module from the Python standard library.

null 1k Sep 23, 2021