A pure-Python KSUID implementation

Related tags

Networking python uuid coordination distributed ulid unique-id ksuid svix
Overview

Svix - Webhooks as a service

Svix-KSUID

API-Lint Frontend-Lint GitHub tag PyPI Join our slack

This library is inspired by Segment's KSUID implementation: https://github.com/segmentio/ksuid

What is a ksuid?

A ksuid is a K sorted UID. In other words, a KSUID also stores a date component, so that ksuids can be approximately sorted based on the time they were created.

Read more here.

Usage

pip install svix-ksuid

from ksuid import Ksuid

ksuid = Ksuid()

Higher timestamp accuracy mode

Ksuids have a 1 second accuracy which is not sufficient for all use-cases. That's why this library exposes a higher accuracy mode which supports accuracy of up to 4ms.

It's fully compatible with normal ksuids, in fact, it outputs valid ksuids. The difference is that it sacrifices one byte of the random payload in favor of this accuracy.

The code too is fully compatible:

pip install svix-ksuid

from ksuid import KsuidMs

ksuid = KsuidMs()

Examples

Default ksuid

Generate a ksuid without passing a specific datetime

7\\xf2up\\x87c\\xad\\xc7tZ\\xf5\\xe7\\xf2'" In [5]: f"Datetime: {ksuid.datetime}" Out[5]: 'Datetime: 2021-05-21 14:04:03' In [6]: f"Timestamp: {ksuid.timestamp}" Out[6]: 'Timestamp: 1621627443' In [7]: f"Payload: {ksuid.payload}" Out[7]: "Payload: b'\\xe1\\x93>7\\xf2up\\x87c\\xad\\xc7tZ\\xf5\\xe7\\xf2'""> 
In [1]: from ksuid import Ksuid

In [2]: ksuid = Ksuid()

In [3]: f"Base62: {ksuid}"
Out[3]: 'Base62: 1srOrx2ZWZBpBUvZwXKQmoEYga2'

In [4]: f"Bytes: {bytes(ksuid)}"
Out[4]: "Bytes: b'\\r5\\xc43\\xe1\\x93>7\\xf2up\\x87c\\xad\\xc7tZ\\xf5\\xe7\\xf2'"

In [5]: f"Datetime: {ksuid.datetime}"
Out[5]: 'Datetime: 2021-05-21 14:04:03'

In [6]: f"Timestamp: {ksuid.timestamp}"
Out[6]: 'Timestamp: 1621627443'

In [7]: f"Payload: {ksuid.payload}"
Out[7]: "Payload: b'\\xe1\\x93>7\\xf2up\\x87c\\xad\\xc7tZ\\xf5\\xe7\\xf2'"

ksuid from datetime

In [1]: datetime = datetime(year=2021, month=5, day=19, hour=1, minute=1, second=1, microsecond=1)

In [2]: datetime
Out[2]: datetime.datetime(2021, 5, 19, 1, 1, 1, 1)

In [3]: ksuid = Ksuid(datetime)

In [4]: ksuid.datetime
Out[4]: datetime.datetime(2021, 5, 19, 1, 1, 1)

In [5]: ksuid.timestamp
Out[5]: 1621407661

ksuid from base62

In [1]: ksuid = Ksuid()

In [2]: ksuid.timestamp
Out[2]: 1621634852

In [3]: f"Base62: {ksuid}"
Out[3]: 'Base62: 1srdszO8Xy2cR6CnARnvxCfRmK4'

In [4]: ksuid_from_base62 = Ksuid.from_base62("1srdszO8Xy2cR6CnARnvxCfRmK4")

In [5]: ksuid_from_base62.timestamp
Out[5]: 1621634852

ksuid from bytes

In [1]: ksuid = Ksuid()

In [2]: ksuid_from_bytes = ksuid.from_bytes(bytes(ksuid))

In [3]: f"ksuid: {ksuid}, ksuid_from_bytes: {ksuid_from_bytes}"
Out[3]: 'ksuid: 1sreAHoz6myPhXghsOdVBoec3Vr, ksuid_from_bytes: 1sreAHoz6myPhXghsOdVBoec3Vr'

In [4]: ksuid == ksuid_from_bytes
Out[4]: True

Compare ksuid(s)

In [1]: ksuid_1 = Ksuid()

In [2]: ksuid_2 = Ksuid.from_bytes(bytes(ksuid_1))

In [3]: f"ksuid_1: {ksuid_1}, ksuid_2: {ksuid_2}"
Out[3]: 'ksuid_1: 1sreAHoz6myPhXghsOdVBoec3Vr, ksuid_2: 1sreAHoz6myPhXghsOdVBoec3Vr'

In [4]: ksuid_1 == ksuid_2
Out[4]: True

In [5]: ksuid_1
Out[5]: 1tM9eRSTrHIrrH5SMEW24rtvIOF

In [6]: ksuid_2
Out[6]: 1tM9eRSTrHIrrH5SMEW24rtvIOF

Order of ksuid(s)

In [1]: ksuid_1 = Ksuid()

In [2]: ksuid_1.timestamp
Out[2]: 1621963256

In [3]: ksuid_2 = Ksuid()

In [4]: ksuid_2.timestamp
Out[4]: 1621963266

In [5]: ksuid_1 < ksuid_2
Out[5]: True

In [6]: ksuid_1 <= ksuid_2
Out[6]: True

In [7]: ksuid_1 >= ksuid_2
Out[7]: False

In [8]: ksuid_1 > ksuid_2
Out[8]: False

License

ksuid source code is available under an MIT License.

Comments
  • ModuleNotFoundError: No module named 'ksuid'

    ModuleNotFoundError: No module named 'ksuid'

    I resolved this, though sharing in case anyone runs into a similar issue. The error could absolutely be due to an unconventional configuration of pip on my system or old pip installation:

    To recreate

    pip3 install svix-ksuid
python3

    Throws ModuleNotFoundError

    Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ksuid import Ksuid
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ksuid'

    Resolution: Use python3 -m pip to install

    pip3 uninstall svix-ksuid -y
python3 -m pip install svix-ksuid 
python3

    Now works as intended

    >>> from ksuid import Ksuid
>>>

    Update

    I later ran into a second issue where, upon attempting to call a python script, the module could not be found:

    <name_of_python_module> <command>
....
....
....
    from ksuid import Ksuid
ModuleNotFoundError: No module named 'ksuid'

    Turns out that I needed to run the installation into the Python installation that my code was referencing using the --target argument. The script ran fine after this

    which python3
# /usr/local/bin/python3
pip3 install --target /usr/local/lib/python3.7/site-packages/ --upgrade svix-ksuid
    opened by kevinggrimm 3
  • String representation of Ksuid(s)

    String representation of Ksuid(s)

    Hello, How exactly do i store these ksuid(s) in db as str? and are these still sortable if they are of type str?

    Also kindly suggest what is the correct way to generate these ids in a web app.

    opened by tausif-deepsource 2
  • Add a mode with increased timestamp accuracy (using some of the payload bytes)

    Add a mode with increased timestamp accuracy (using some of the payload bytes)

    We can use one byte (4ms accuracy), or two bytes (much more) from the payload for a fixed-point decimal portion for the timestamp. This will increase accuracy and will keep it fully compatible with ksuids.

    I think 4ms is sufficient, as the accuracy of time-sync and timestamps at a higher resolution is probably questionable. We can always extend it later on if needed, though for our use-cases it's probably more than enough.

    opened by tasn 1
  • Fix Ksuid comparisons.

    Fix Ksuid comparisons.

    We were comparing based only on the timestamp which was breaking ordering and was also internally inconsistent because you could have two uids that are not equal, but also not greater or lesser than each other.

    Fixes #1

    opened by tasn 0
  • Comparison is wrong - it doesn't use the payload

    Comparison is wrong - it doesn't use the payload

    Two different ksuids with the same timestamp (but different payloads) will not be equal, but will also not be different when doing comparisons (lt/gt). This is due to the __lt__ function not comparing the whole byte array, but rather just the datetime. It should compare the whole byte array.

    opened by tasn 0
Owner
Svix
Webhooks as a service
Svix
GitHub https://www.svix.com
Pywbem - A WBEM client and related utilities, written in pure Python.

Pywbem - A WBEM client and related utilities, written in pure Python Overview Pywbem is a WBEM client and WBEM indication listener and provides relate

PyWBEM Projects 39 Dec 22, 2022
msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+

msgspec msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+. In addition to serialization/deserializat

Jim Crist-Harif 414 Jan 6, 2023
telnet implementation over TCP socket with python

This a P2P implementation of telnet. This program transfers data on TCP sockets as plain text

null 10 May 19, 2022
Python implementation of the IPv8 layer provide authenticated communication with privacy

Python implementation of the IPv8 layer provide authenticated communication with privacy

null 203 Oct 26, 2022
Python implementation of the Session open group server

API Documentation CLI Reference Want to build from source? See BUILDING.md. Want to deploy using Docker? See DOCKER.md. Installation Instructions Vide

Oxen 36 Jan 2, 2023
QUIC and HTTP/3 implementation in Python

aioquic What is aioquic? aioquic is a library for the QUIC network protocol in Python. It features a minimal TLS 1.3 implementation, a QUIC stack and

null 1.2k Dec 29, 2022
BaseSpec is a system that performs a comparative analysis of baseband implementation and the specifications of cellular networks.

BaseSpec is a system that performs a comparative analysis of baseband implementation and the specifications of cellular networks. The key intuition of BaseSpec is that a message decoder in baseband software embeds the protocol specification in a machine-friendly structure to parse incoming messages;

SysSec Lab 35 Dec 6, 2022
A Scapy implementation of SMS-SUBMIT and (U)SIM Application Toolkit command packets.

A Scapy implementation of SMS-SUBMIT and (U)SIM Application Toolkit command packets.

mnemonic 83 Dec 11, 2022
The Delegate Network: An Interactive Voice Response Delegative Democracy Implementation of Liquid Democracy

The Delegate Network Overview The delegate network is a completely transparent, easy-to-use and understand version of what is sometimes called liquid

James Bowery 2 Feb 25, 2022
netpy - more than implementation of netcat 🐍🔥

netpy - more than implementation of netcat ????

Mahmoud S. ElGammal 1 Jan 26, 2022
A simple implementation of an RPC toolkit

Simple RPC With Raw Sockets Repository for the Data network course project: Introduction In this project, you will attempt to code a simple implementa

Milad Samimifar 1 Mar 25, 2022
Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7.

Medusa Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7. Installation To install Medusa, you'll need Mythic installed o

Mythic Agents 123 Nov 9, 2022
ProtOSINT is a Python script that helps you investigate Protonmail accounts and ProtonVPN IP addresses

ProtOSINT ProtOSINT is a Python script that helps you investigate ProtonMail accounts and ProtonVPN IP addresses. Description This tool can help you i

pixelbubble 249 Dec 23, 2022
A Python tool used to automate the execution of the following tools : Nmap , Nikto and Dirsearch but also to automate the report generation during a Web Penetration Testing

?? WebMap A Python tool used to automate the execution of the following tools : Nmap , Nikto and Dirsearch but also to automate the report generation

Iliass Alami Qammouri 274 Jan 1, 2023
Light, simple RPC framework for Python

Agileutil是一个Python3 RPC框架。基于微服务架构，封装了rpc/http/orm/log等常用组件，提供了简洁的API，开发者可以很快上手，快速进行业务开发。

null 16 Nov 22, 2022
Minimal, self-hosted, 0-config alternative to ngrok. Caddy+OpenSSH+50 lines of Python.

If you have a webserver running on one computer (say your development laptop), and you want to expose it securely (ie HTTPS) via a public URL, SirTunnel allows you to easily do that.

Anders Pitman 423 Jan 2, 2023
NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.

NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.

NetworkX 12k Jan 2, 2023
A Python library to ease the integration with the Beem Africa (SMS, AIRTIME, OTP, 2WAY-SMS, BPAY, USSD)

python-client A Python library to easy the integration with the Beem Africa SMS Gateway Features to be Implemented Airtime OTP SMS Two way SMS USSD Bp

Beem Africa 24 Oct 29, 2022
Python port of proxy-www (https://github.com/justjavac/proxy-www)

proxy-www.py Python port of proxy-www (https://github.com/justjavac/proxy-www). Implemented additional functionalities! How to install pip install pro

Minjun Kim (Lapis0875) 20 Dec 8, 2021