Python collections that are backended by sqlite3 DB and are compatible with the built-in collections

Related tags

Data Structures sqlitecollections
Overview

sqlitecollections

Python collections that are backended by sqlite3 DB and are compatible with the built-in collections

Installation

$ pip install git+ssh://[email protected]/osoken/sqlitecollections.git

Development

To run tests, type checking and linting locally, we use tox. It will run pytest, mypy and black on python 3.6, 3.7, 3.8 and 3.9. Install them via the following commands:

$ git clone [email protected]:osoken/sqlitecollections.git
$ cd sqlitecollections
$ python -m venv .venv
$ source ./.venv/bin/activate
$ pip install -e .[dev]

then, run tests:

$ tox

Compatibility policy

We aim to implement containers that are as compatible as possible with the built-in containers, but we have a few implementations that intentionally behave differently.

  • Any member in the container cannot be mutated directly. If you want to mutate any member, mutate it via temporary variable then write it back.
from sqlitecollections import Dict

x = Dict(a=[]) # create {"a": []}
x["a"].append("b")  # try to mutate the empty list
print(x["a"])  # not ["b"] but []

temp = x["a"]  # temporarily substitute the list to a variable
temp.append("b")  # mutate the temporary variable
x["a"] = temp  # then, write it back
print(x["a"])  # now, we get ["b"]
  • Dict's item order is guaranteed to be insertion order not only for python 3.7 and upper but for all versions.
  • fromkeys class method is not provided.
Comments
  • divide benchmarking process and rendering process

    divide benchmarking process and rendering process

    Is your feature request related to a problem? Please describe.

    currently, scbenchmarker module executes benchmaring then rendering. regarding #232, we need a way to partially update the result.

    Describe the solution you'd like

    add subcommand to scbenchmarker, run and render, and divide current process into execution phase and render phase.

    Describe alternatives you've considered

    Additional context

    documentation enhancement 
    opened by osoken 1
  • PR template doesn't work

    PR template doesn't work

    Describe the bug The PR template you placed simply doesn't work.

    To Reproduce Just try to create a PR using any branch with some changes.

    Expected behavior The PR template you placed shows up.

    Additional context None

    bug 
    opened by osoken 1
  • Remove RebuildStrategy and rebuild process on initialization (breaking change on v1.0 release)

    Remove RebuildStrategy and rebuild process on initialization (breaking change on v1.0 release)

    Specify pickle protocol version of default serializer. The protocol version 4 suits our needs.

    Thank you, @hanxiao, for bringing this to my attention.

    enhancement 
    opened by osoken 1
  • docarray support

    docarray support

    Hi Takeshi, I found this repo very interesting. I'm currently working on sqlite backend for https://github.com/jina-ai/docarray Would be nice to keep in touch and get your review on it.

    I'm also referring https://github.com/RaRe-Technologies/sqlitedict

    opened by hanxiao 1
  • Set.issubset is very slow

    Set.issubset is very slow

    Improve the time-efficiency. The rough benchmark for Set.issubset for 100000 elements set and 100000 elements iterable is as follows:

    {'subject': '`issubset`', 'one': {'name': '`set`', 'timing': 0.01689779758453369, 'memory': 6.0}, 'another': {'name': '`sqlitecollections.set`', 'timing': 453.76503455638885, 'memory': 0.0078125}, 'ratio': {'timing': 26853.501604949593, 'memory': 0.0013020833333333333}}
    enhancement 
    opened by osoken 1
  • Feature/issue 271 support python311

    Feature/issue 271 support python311

    Description

    python 3.11rc2 -> 3.11.

    Fixes #271

    Type of change

    Please delete options that are not relevant.

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [x] This change requires a documentation update
    • [ ] Performance Improvement (non-breaking change which improves performance of some function)

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [x] I have added tests that prove my fix is effective or that my feature works
    opened by osoken 0
  • add support for python 3.11

    add support for python 3.11

    Is your feature request related to a problem? Please describe.

    Python 3.11 is released, so it's time to get rid of tests on python 3.11 rc2.

    Describe the solution you'd like

    remove rc2 from tox.ini and pipeline files.

    Describe alternatives you've considered None

    Additional context None

    enhancement 
    opened by osoken 0
  • Feature/issue 269 add python11 support

    Feature/issue 269 add python11 support

    Description

    Add test for python 3.11 and fix errors.

    Fixes #269

    Type of change

    Please delete options that are not relevant.

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [x] This change requires a documentation update
    • [ ] Performance Improvement (non-breaking change which improves performance of some function)

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [x] I have added tests that prove my fix is effective or that my feature works
    opened by osoken 0
  • python 11 support

    python 11 support

    Is your feature request related to a problem? Please describe.

    The current code base only supports up to python 3.10.

    Describe the solution you'd like Add tests and type checks for python 3.11.

    Describe alternatives you've considered

    None

    Additional context

    None

    opened by osoken 0
  • Feat/issue 267 support sort strategy

    Feat/issue 267 support sort strategy

    Description

    add support for SortingStrategy so that users can select which sorting algorithm to be used.

    Fixes #267

    Type of change

    Please delete options that are not relevant.

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [x] This change requires a documentation update
    • [x] Performance Improvement (non-breaking change which improves performance of some function)

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [x] I have added tests that prove my fix is effective or that my feature works
    opened by osoken 0
  • Support sort strategy

    Support sort strategy

    Is your feature request related to a problem? Please describe.

    There are strategies in List.sort that optimize memory and computation time, respectively, and strategies in between. Memory-optimized strategy will be very slow but there might be a case that the limited memory doesn't allow to hold even the array of indices of the list.

    Describe the solution you'd like

    Add SortingStrategy to control which strategy is used when sorting.

    Describe alternatives you've considered

    Additional context

    documentation enhancement 
    opened by osoken 0
  • Add security policy

    Add security policy

    Is your feature request related to a problem? Please describe. There is no security policy files in this repository.

    Describe the solution you'd like Simply add the security policy file.

    Describe alternatives you've considered None

    Additional context None

    opened by osoken 0
  • add cached Set

    add cached Set

    Is your feature request related to a problem? Please describe.

    Current implementation strictly syncs the contents and the DB state. It helps keeping the consistency but causes some performance problem.

    Describe the solution you'd like Add cached version of the List container.

    Describe alternatives you've considered

    Additional context

    enhancement 
    opened by osoken 0
  • add cached Dict

    add cached Dict

    Is your feature request related to a problem? Please describe.

    Current implementation strictly syncs the contents and the DB state. It helps keeping the consistency but causes some performance problem.

    Describe the solution you'd like Add cached version of the Dict container.

    Describe alternatives you've considered

    Additional context

    enhancement 
    opened by osoken 0
  • add cached version of List

    add cached version of List

    Is your feature request related to a problem? Please describe.

    Current implementation strictly syncs the contents and the DB state. It helps keeping the consistency but causes some performance problem.

    Describe the solution you'd like

    Add cached version of the List container.

    Describe alternatives you've considered

    Additional context

    enhancement 
    opened by osoken 0
  • KeysView doesn't raise RuntimeError when dict length changed.

    KeysView doesn't raise RuntimeError when dict length changed.

    Expected behavior:

    >>> import sqlitecollections as sc
>>> x =  sc.Dict({"A": 1, "B": 2, "C": 3})
>>> for k in x.keys():
...     print(k)
...     x["d"] = 12
... 
A
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

    Actual behavior on v0.9.5

    >>> for k in x.keys():
...     print(k)
...     x["d"] = 12
... 
A
B
C
d
>>>
    bug enhancement 
    opened by osoken 0
Owner
Takeshi OSOEKAWA
Takeshi OSOEKAWA
GitHub
An esoteric data type built entirely of NaNs.

NaNsAreNumbers An esoteric data type built entirely of NaNs. Installation pip install nans_are_numbers Explanation A floating point number is just co

Travis Hoppe 72 Jan 1, 2023
My notes on Data structure and Algos in golang implementation and python

My notes on DS and Algo Table of Contents Arrays LinkedList Trees Types of trees: Tree/Graph Traversal Algorithms Heap Priorty Queue Trie Graphs Graph

Chia Yong Kang 0 Feb 13, 2022
This repository is a compilation of important Data Structures and Algorithms based on Python.

Python DSA ?? This repository is a compilation of important Data Structures and Algorithms based on Python. Please make seperate folders for different

Bhavya Verma 27 Oct 29, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 1, 2023
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Basic sort and search algorithms written in python.

Basic sort and search algorithms written in python. These were all developed as part of my Computer Science course to demonstrate understanding so they aren't 100% efficent

Ben Jones 0 Dec 14, 2022
A DSA repository but everything is in python.

DSA Status Contents A: Mathematics B: Bit Magic C: Recursion D: Arrays E: Searching F: Sorting G: Matrix H: Hashing I: String J: Linked List K: Stack

Shubhashish Dixit 63 Dec 23, 2022
Common sorting algorithims in Python

This a Github Repository with code for my attempts for commonly used sorting algorithims, tested on a list with 3000 randomly generated numbers.

Pratham Prasoon 14 Sep 2, 2021
pyprobables is a pure-python library for probabilistic data structures

pyprobables is a pure-python library for probabilistic data structures. The goal is to provide the developer with a pure-python implementation of common probabilistic data-structures to use in their work.

Tyler Barrus 86 Dec 25, 2022
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022
nocasedict - A case-insensitive ordered dictionary for Python

nocasedict - A case-insensitive ordered dictionary for Python Overview Class NocaseDict is a case-insensitive ordered dictionary that preserves the or

PyWBEM Projects 2 Dec 12, 2021
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
A Python library for electronic structure pre/post-processing

PyProcar PyProcar is a robust, open-source Python library used for pre- and post-processing of the electronic structure data coming from DFT calculati

Romero Group 124 Dec 7, 2022
Programming of a spanning tree algorithm with Python : In depth first with a root node.

ST Algorithm Programming of a spanning tree algorithm with Python : In depth first with a root node. Description This programm reads informations abou

Mathieu Lamon 1 Dec 16, 2021
A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

faas-cache-dict A Python dictionary implementation designed to act as an in-memory cache for FaaS environments. Formally you would describe this a mem

Juan 3 Dec 13, 2022
Data Structure With Python

Data-Structure-With-Python- Python programs also include in this repo Stack A stack is a linear data structure that stores items in a Last-In/First-Ou

Sumit Nautiyal 2 Jan 9, 2022
🔬 Fixed struct serialization system, using Python 3.9 annotated type hints

py-struct Fixed-size struct serialization, using Python 3.9 annotated type hints This was originally uploaded as a Gist because it's not intended as a

Alba Mendez 4 Jan 14, 2022
A Python implementation of red-black trees

Python red-black trees A Python implementation of red-black trees. This code was originally copied from programiz.com, but I have made a few tweaks to

Emily Dolson 7 Oct 20, 2022
This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages

DSA-Code-Snippet This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages Cont

DSCSRMNCR 3 Oct 22, 2021