import pickledb
db = pickledb.load('example.db', False)
db.set('key', 'value')
db.dump()
db = pickledb.load('example.db', False)

Traceback (most recent call last): File "", line 1, in File "C:\Users\Documents\Python Projects\garbage-chatter\venv\lib\site-packages\pickledb.py", line 39, in load return pickledb(location, auto_dump) File "C:\Users\Documents\Python Projects\garbage-chatter\venv\lib\site-packages\pickledb.py", line 49, in init self.load(location, auto_dump) File "C:\Users\Documents\Python Projects\garbage-chatter\venv\lib\site-packages\pickledb.py", line 79, in load self.loaddb() File "C:\Users\Documents\Python Projects\garbage-chatter\venv\lib\site-packages\pickledb.py", line 96, in loaddb self.db = json.load(open(self.loco, 'rb')) File "C:\Users\AppData\Local\Programs\Python\Python35\Lib\json_init.py", line 268, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "C:\Users\AppData\Local\Programs\Python\Python35\Lib\json_init.py", line 312, in loads s.class.name)) TypeError: the JSON object must be str, not 'bytes'

as diagnosed by maintainers here; https://github.com/patx/pickledb/issues/37

this shows the user is supplying an int and the framework changes the type to string.

JSON supports integer tyes as values and talking about JSON “values” confuses this discussion on a key/value pair database where the key type is being discovered, not the value.

When a SIGTERM signal (e.g. by hitting Ctrl-C in terminal) is received during dump() execution, the exit is performed immediately which causes corruption of the pickledb database file. This can be very annoying as it might mean the loss of critical data.

Upon execution of load() after a restart, I get this this error:

Traceback (most recent call last):
  File "test.py", line 22, in <module>
    main()
  File "test.py", line 14, in main
    db = get_pickledb()
  File "test.py", line 9, in get_pickledb
    db = pickledb.load(DB_FILEPATH, False)
  File "/home/don/tmp/pickledb_robust/pickledb/pickledb.py", line 34, in load
    return pickledb(location, option)
  File "/home/don/tmp/pickledb_robust/pickledb/pickledb.py", line 41, in __init__
    self.load(location, option)
  File "/home/don/tmp/pickledb_robust/pickledb/pickledb.py", line 49, in load
    self._loaddb()
  File "/home/don/tmp/pickledb_robust/pickledb/pickledb.py", line 186, in _loaddb
    self.db = json.load(open(self.loco, 'rb'))
  File "/usr/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

My pull request will fix this: https://github.com/patx/pickledb/pull/17

import pickledb

db = pickledb.load('database.sql', True)
db.set('key', ['one', 'two', 'three'])

db.ladd('key', 'four')

db.lremvalue('key', 'four') # You need to use db.dump() as a workaround
# db.dump()

To anyone who might have stumbled upon this package.. as far as my understanding goes, pickleDB won't work for large dictionaries, since it keeps the entire dictionary in RAM (or loads the entire db-file at once using simplejson). So, if you are looking to work with very large disk-based dictionaries, you might be better off using the Sqlite database, and writing a dictionary like wrapper on top.

Also, the Shelve module seems to work nicely for large dictionaries. However, it has the downside that it uses Pickle internally, and thus, the Database files are not directly readable.

Please correct me if I am wrong. Thanks!

There seems to be an issue with the example provided by pickledb itself. When calling the dump function with Python 3, there is a TypeError produced by the code.

Here's what was inputted in a test file:

import pickledb
db = pickledb.load("test.db", False)
db.set("key", "value")
print(db.get("key"))
db.dump()

Traceback:

Traceback (most recent call last):
  File "tests.py", line 6, in <module>
    db.dump()
  File "/Users/shreydesai/GitHub/pickledb/pickledb.py", line 56, in dump
    self._dumpdb(True)
  File "/Users/shreydesai/GitHub/pickledb/pickledb.py", line 201, in _dumpdb
    simplejson.dump(self.db, open(self.loco, 'wb'))
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/
simplejson/__init__.py", line 277, in dump
    fp.write(chunk)

TypeError: 'str' does not support the buffer interface

I am thinking to write a web api wrapper around pickledb so that it can be accessed over http among multiple servers example : http://someurl:8080/set/a post : value

http://someurl:8080/get/a returns its value in JSOiN format

So I wanted to ask community its worth the effort or not.

Full error message:

File "**hidden**/Functions/Main/m.py", line 32, in m
    db.dump()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pickledb.py", line 92, in dump
    json.dump(self.db, open(self.loco, 'wt'))
OSError: [Errno 30] Read-only file system: '../../database.txt

Code used: db = pickledb.load('../../database.txt', False)

I'm running a simple example, and pickledb seems to successfully set the value of key1 and prints it right after setting it. However, the pickle.db file is not created and attempting to get the value fails on the next run.

import pickledb

db = pickledb.load('pickle.db', False)

db.set('key1', 'value')

print(db.get('key1'))

$ python3 pickledb_test.py
value

$ ls
pickledb_test.py

Using Python 3.8.5 and pickleDB 0.9.2.

Edit: This was fixed by enabling auto_dump.

pickledb.load('pickle.db', True)

Hi, thank you for reading this. What do you think about this idea to switch from an ordinary dictionary to the OrderedDict? I find that most of the times in my projects I need data to be sorted by keys so I have to do it manually. I thought maybe I'm not the only one and we can all benefit from making it a built-in feature? :) OrderedDict is located in the "collections" std-lib module.

It would be awesome to have optional encryption and decryption function supplied by the user so that they can encrypt the data before writing to file or decrypt using the decryptor function before reading from given file.

Hello,

I'm getting JSON errors, and I'm not sure if I'm doing something wrong, or if its a bug.

I'm getting this error on 3 different machines (2 Ubuntu, 1 Windows). One Ubuntu and Windows use python3.9, the other Ubuntu python3.8. All PickleDB's modules have been installed today via pip.

Code:

#/usr/bin/python3
import pickledb

db = pickledb.load('pik.db', False)
db.set('foo', 'bar')

print(db.get('foo'))

When I run it:

ubuntu@mimas:~/pik$ python3 pik.py 
Traceback (most recent call last):
  File "pik.py", line 5, in <module>
    db = pickledb.load('pik.db', False)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pickledb.py", line 43, in load
    return PickleDB(location, auto_dump, sig)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pickledb.py", line 54, in __init__
    self.load(location, auto_dump)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pickledb.py", line 85, in load
    self._loaddb()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pickledb.py", line 102, in _loaddb
    self.db = json.load(open(self.loco, 'rt'))
  File "/usr/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Like to hear from you guys.

In order to ensure atomicity, dump the db into a temporary file before moving to the correct location. This will avoid creating empty file, mostly when process catch a SIGTERM signal.

Also, clean up the extra dump done just before doing the same via a thread. This should reduce write overhead.

In the pickledb.py dump function(code line 90), the json.dump() func dumps data the first time, then the dthread dumps data second time.

It seems a little confusing. Is that a bug or does anybody know why?