Kartothek - a Python library to manage large amounts of tabular data in a blob store

Overview

Kartothek

Build Status Documentation Status codecov.io License: MIT Anaconda-Server Badge Anaconda-Server Badge

Kartothek is a Python library to manage (create, read, update, delete) large amounts of tabular data in a blob store. It stores data as datasets, which it presents as pandas DataFrames to the user. Datasets are a collection of files with the same schema that reside in a blob store. Kartothek uses a metadata definition to handle these datasets efficiently. For distributed access and manipulation of datasets Kartothek offers a Dask interface.

Storing data distributed over multiple files in a blob store (S3, ABS, GCS, etc.) allows for a fast, cost-efficient and highly scalable data infrastructure. A downside of storing data solely in an object store is that the storages themselves give little to no guarantees beyond the consistency of a single file. In particular, they cannot guarantee the consistency of your dataset. If we demand a consistent state of our dataset at all times, we need to track the state of the dataset. Kartothek frees us from having to do this manually.

The kartothek.io module provides building blocks to create and modify these datasets in data pipelines. Kartothek handles I/O, tracks dataset partitions and selects subsets of data transparently.

Installation

Installers for the latest released version are availabe at the Python package index and on conda.

# Install with pip
pip install kartothek
# Install with conda
conda install -c conda-forge kartothek

What is a (real) Kartothek?

A Kartothek (or more modern: Zettelkasten/Katalogkasten) is a tool to organize (high-level) information extracted from a source of information.

Comments
  • Update packaging setup

    Update packaging setup

    Description:

    Use declarative packaging, move to environment.yml and limit supported Python versions to 3.8-3.10.

    • [ ] ~Closes #xxxx~
    • [x] Changelog entry
    opened by xhochy 1
  • Bump actions/cache from 3.0.8 to 3.0.9

    Bump actions/cache from 3.0.8 to 3.0.9

    Bumps actions/cache from 3.0.8 to 3.0.9.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.9

    • Enhanced the warning message for cache unavailability in case of GHES.
    Changelog

    Sourced from actions/cache's changelog.

    3.0.8

    • Fix zstd not working for windows on gnu tar in issues #888 and #891.
    • Allowing users to provide a custom timeout as input for aborting download of a cache segment using an environment variable SEGMENT_DOWNLOAD_TIMEOUT_MIN. Default is 60 minutes.

    3.0.9

    • Enhanced the warning message for cache unavailablity in case of GHES.
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump codecov/codecov-action from 3.1.0 to 3.1.1

    Bump codecov/codecov-action from 3.1.0 to 3.1.1

    Bumps codecov/codecov-action from 3.1.0 to 3.1.1.

    Release notes

    Sourced from codecov/codecov-action's releases.

    3.1.1

    What's Changed

    New Contributors

    Full Changelog: https://github.com/codecov/codecov-action/compare/v3.1.0...v3.1.1

    Changelog

    Sourced from codecov/codecov-action's changelog.

    3.1.1

    Fixes

    • #661 Update deprecation warning
    • #593 Create codeql-analysis.yml
    • #712 README: fix typo
    • #725 fix: Remove a blank row
    • #726 Update README.md with correct badge version
    • #633 Create scorecards-analysis.yml
    • #747 fix: add more verbosity to validation
    • #750 Regenerate scorecards-analysis.yml
    • #774 Switch to v3
    • #783 Fix network entry in table
    • #791 Trim arguments after splitting them
    • #769 Plumb failCi into verification function.

    Dependencies

    • #713 build(deps-dev): bump typescript from 4.6.3 to 4.6.4
    • #714 build(deps): bump node-fetch from 3.2.3 to 3.2.4
    • #724 build(deps): bump github/codeql-action from 1 to 2
    • #717 build(deps-dev): bump @​types/jest from 27.4.1 to 27.5.0
    • #729 build(deps-dev): bump @​types/node from 17.0.25 to 17.0.33
    • #734 build(deps-dev): downgrade @​types/node to 16.11.35
    • #723 build(deps): bump actions/checkout from 2 to 3
    • #733 build(deps): bump @​actions/github from 5.0.1 to 5.0.3
    • #732 build(deps): bump @​actions/core from 1.6.0 to 1.8.2
    • #737 build(deps-dev): bump @​types/node from 16.11.35 to 16.11.36
    • #749 build(deps): bump ossf/scorecard-action from 1.0.1 to 1.1.0
    • #755 build(deps-dev): bump typescript from 4.6.4 to 4.7.3
    • #759 build(deps-dev): bump @​types/node from 16.11.36 to 16.11.39
    • #762 build(deps-dev): bump @​types/node from 16.11.39 to 16.11.40
    • #746 build(deps-dev): bump @​vercel/ncc from 0.33.4 to 0.34.0
    • #757 build(deps): bump ossf/scorecard-action from 1.1.0 to 1.1.1
    • #760 build(deps): bump openpgp from 5.2.1 to 5.3.0
    • #748 build(deps): bump actions/upload-artifact from 2.3.1 to 3.1.0
    • #766 build(deps-dev): bump typescript from 4.7.3 to 4.7.4
    • #799 build(deps): bump openpgp from 5.3.0 to 5.4.0
    • #798 build(deps): bump @​actions/core from 1.8.2 to 1.9.1
    Commits
    • d9f34f8 release: update changelog and version to 3.1.1 (#828)
    • 0e9e7b4 Plumb failCi into verification function. (#769)
    • 7f20bd4 build(deps): bump @​actions/core from 1.8.2 to 1.9.1 (#798)
    • 13bc253 build(deps): bump openpgp from 5.3.0 to 5.4.0 (#799)
    • 5c0da1b Trim arguments after splitting them (#791)
    • 68d5f6d Fix network entry in table (#783)
    • 2a829b9 Switch to v3 (#774)
    • 8e09eaf build(deps-dev): bump typescript from 4.7.3 to 4.7.4 (#766)
    • 39e2229 build(deps): bump actions/upload-artifact from 2.3.1 to 3.1.0 (#748)
    • b2b7703 build(deps): bump openpgp from 5.2.1 to 5.3.0 (#760)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/cache from 3.0.2 to 3.0.7

    Bump actions/cache from 3.0.2 to 3.0.7

    Bumps actions/cache from 3.0.2 to 3.0.7.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.7

    What's Changed

    • Fix for the download stuck problem has been added in actions/cache for users who were intermittently facing the issue. As part of this fix, new timeout has been introduced in the download step to stop the download if it doesn't complete within an hour and run the rest of the workflow without erroring out.

    v3.0.6

    What's Changed

    • Add example for clojure lein project dependencies by @​shivamarora1 in PR actions/cache#835
    • Update toolkit's cache npm module to latest. Bump cache version to v3.0.6 by @​pdotl in PR actions/cache#887
    • Fix issue #809 where cache save/restore was failing for Amazon Linux 2 runners due to older tar version
    • Fix issue #833 where cache save was not working for caching github workspace directory

    New Contributors

    Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.6

    v3.0.5

    Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit.

    v3.0.4

    In this release, we have fixed the tar creation error while trying to create it with path as ~/ home folder on ubuntu-latest.

    v3.0.3

    Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

    Changelog

    Sourced from actions/cache's changelog.

    3.0.2

    • Added support for dynamic cache size cap on GHES.

    3.0.3

    • Fixed avoiding empty cache save when no files are available for caching. (issue)

    3.0.4

    • Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)

    3.0.5

    • Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)

    3.0.6

    • Fixed #809 - zstd -d: no such file or directory error
    • Fixed #833 - cache doesn't work with github workspace directory

    3.0.7

    • Fixed #810 - download stuck issue. A new timeout is introduced in the download process to abort the download if it gets stuck and doesn't finish within an hour.
    Commits
    • a7c34ad Merge pull request #894 from actions/kotewar/update-toolkit-version
    • 83394c9 Updated cache version in license file
    • e839c25 Updated actions/cache version to 3.0.3
    • 33a923d Added release information
    • a404368 Updated actions/cache version to 3.0.2
    • f427802 Merge pull request #887 from actions/pdotl-version-patch
    • 9916fe1 Update cache version in licences
    • 318935e Update README and RELEASES
    • 85efbb5 Update cache npm module to latest
    • 4387dbc Merge pull request #835 from shivamarora1/clojure-lein-example
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/cache from 3.0.2 to 3.0.6

    Bump actions/cache from 3.0.2 to 3.0.6

    Bumps actions/cache from 3.0.2 to 3.0.6.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.6

    What's Changed

    • Add example for clojure lein project dependencies by @​shivamarora1 in PR actions/cache#835
    • Update toolkit's cache npm module to latest. Bump cache version to v3.0.6 by @​pdotl in PR actions/cache#887
    • Fix issue #809 where cache save/restore was failing for Amazon Linux 2 runners due to older tar version
    • Fix issue #833 where cache save was not working for caching github workspace directory

    New Contributors

    Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.6

    v3.0.5

    Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit.

    v3.0.4

    In this release, we have fixed the tar creation error while trying to create it with path as ~/ home folder on ubuntu-latest.

    v3.0.3

    Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

    Changelog

    Sourced from actions/cache's changelog.

    3.0.2

    • Added support for dynamic cache size cap on GHES.

    3.0.3

    • Fixed avoiding empty cache save when no files are available for caching. (issue)

    3.0.4

    • Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)

    3.0.5

    • Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)

    3.0.6

    • Fixed #809 - zstd -d: no such file or directory error
    • Fixed #833 - cache doesn't work with github workspace directory
    Commits
    • f427802 Merge pull request #887 from actions/pdotl-version-patch
    • 9916fe1 Update cache version in licences
    • 318935e Update README and RELEASES
    • 85efbb5 Update cache npm module to latest
    • 4387dbc Merge pull request #835 from shivamarora1/clojure-lein-example
    • 71e3ee5 Add phantsure to PR reviewers
    • c316eb7 Add phantsure to issue assignment
    • 0865c47 new release (#855)
    • 354a2ae Consuming 3.0 actions/cache (#834)
    • baed351 Merge pull request #530 from axelson/document-where-to-cache
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/cache from 3.0.2 to 3.0.5

    Bump actions/cache from 3.0.2 to 3.0.5

    Bumps actions/cache from 3.0.2 to 3.0.5.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.5

    Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit.

    v3.0.4

    In this release, we have fixed the tar creation error while trying to create it with path as ~/ home folder on ubuntu-latest.

    v3.0.3

    Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

    Changelog

    Sourced from actions/cache's changelog.

    3.0.2

    • Added support for dynamic cache size cap on GHES.

    3.0.3

    • Fixed avoiding empty cache save when no files are available for caching. (issue)

    3.0.4

    • Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)

    3.0.5

    • Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)
    Commits
    • 0865c47 new release (#855)
    • 354a2ae Consuming 3.0 actions/cache (#834)
    • baed351 Merge pull request #530 from axelson/document-where-to-cache
    • 8829e97 Update README.md
    • eec8cd3 Merge pull request #836 from actions/vsvipul/fix-auto-assign
    • 5cc84c0 Add kotewar and remove phantsure from auto-assignees lists
    • afc669e Merge pull request #819 from mpilgrem/haskell-stack-example
    • a0efc56 Use pull_request_target instead of pull_request
    • d25c51b Adapt existing Haskell Stack example for Windows
    • a080a3b Merge pull request #816 from lobis/main
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/setup-python from 3 to 4.1.0

    Bump actions/setup-python from 3 to 4.1.0

    Bumps actions/setup-python from 3 to 4.1.0.

    Release notes

    Sourced from actions/setup-python's releases.

    v4.1.0

    In scope of this pull request we updated actions/cache package as the new version contains fixes for caching error handling. Moreover, we added a new input update-environment. This option allows to specify if the action shall update environment variables (default) or not.

    Update-environment input

        - name: setup-python 3.9
          uses: actions/setup-python@v4
          with:
            python-version: 3.9
            update-environment: false
    

    Besides, we added such changes as:

    v4.0.0

    What's Changed

    • Support for python-version-file input: #336

    Example of usage:

    - uses: actions/setup-python@v4
      with:
        python-version-file: '.python-version' # Read python version from a file
    - run: python my_script.py
    

    There is no default python version for this setup-python major version, the action requires to specify either python-version input or python-version-file input. If the python-version input is not specified the action will try to read required version from file from python-version-file input.

    • Use pypyX.Y for PyPy python-version input: #349

    Example of usage:

    - uses: actions/setup-python@v4
      with:
        python-version: 'pypy3.9' # pypy-X.Y kept for backward compatibility
    - run: python my_script.py
    
    • RUNNER_TOOL_CACHE environment variable is equal AGENT_TOOLSDIRECTORY: #338

    • Bugfix: create missing pypyX.Y symlinks: #347

    • PKG_CONFIG_PATH environment variable: #400

    • Added python-path output: #405

    ... (truncated)

    Commits
    • c4e89fa Improve readme for 3.x and 3.11-dev style python-version (#441)
    • 0ad0f6a Merge pull request #452 from mayeut/fix-env
    • f0bcf8b Merge pull request #456 from akx/patch-1
    • af97157 doc: Add multiple wildcards example to readme
    • 364e819 Merge pull request #394 from akv-platform/v-sedoli/set-env-by-default
    • 782f81b Merge pull request #450 from IvanZosimov/ResolveVersionFix
    • 2c9de4e Remove duplicate code introduced in #440
    • 412091c Fix tests for update-environment==false
    • 78a2330 Merge pull request #451 from dmitry-shibanov/fx-pipenv-python-version
    • 96f494e trigger checks
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/setup-python from 3 to 4

    Bump actions/setup-python from 3 to 4

    Bumps actions/setup-python from 3 to 4.

    Release notes

    Sourced from actions/setup-python's releases.

    v4.0.0

    What's Changed

    • Support for python-version-file input: #336

    Example of usage:

    - uses: actions/setup-python@v4
      with:
        python-version-file: '.python-version' # Read python version from a file
    - run: python my_script.py
    

    There is no default python version for this setup-python major version, the action requires to specify either python-version input or python-version-file input. If the python-version input is not specified the action will try to read required version from file from python-version-file input.

    • Use pypyX.Y for PyPy python-version input: #349

    Example of usage:

    - uses: actions/setup-python@v4
      with:
        python-version: 'pypy3.9' # pypy-X.Y kept for backward compatibility
    - run: python my_script.py
    
    • RUNNER_TOOL_CACHE environment variable is equal AGENT_TOOLSDIRECTORY: #338

    • Bugfix: create missing pypyX.Y symlinks: #347

    • PKG_CONFIG_PATH environment variable: #400

    • Added python-path output: #405 python-path output contains Python executable path.

    • Updated zeit/ncc to vercel/ncc package: #393

    • Bugfix: fixed output for prerelease version of poetry: #409

    • Made pythonLocation environment variable consistent for Python and PyPy: #418

    • Bugfix for 3.x-dev syntax: #417

    • Other improvements: #318 #396 #384 #387 #388

    Update actions/cache version to 2.0.2

    In scope of this release we updated actions/cache package as the new version contains fixes related to GHES 3.5 (actions/setup-python#382)

    Add "cache-hit" output and fix "python-version" output for PyPy

    This release introduces new output cache-hit (actions/setup-python#373) and fix python-version output for PyPy (actions/setup-python#365)

    The cache-hit output contains boolean value indicating that an exact match was found for the key. It shows that the action uses already existing cache or not. The output is available only if cache is enabled.

    ... (truncated)

    Commits
    • d09bd5e fix: 3.x-dev can install a 3.y version (#417)
    • f72db17 Made env.var pythonLocation consistent for Python and PyPy (#418)
    • 53e1529 add support for python-version-file (#336)
    • 3f82819 Fix output for prerelease version of poetry (#409)
    • 397252c Update zeit/ncc to vercel/ncc (#393)
    • de977ad Merge pull request #412 from vsafonkin/v-vsafonkin/fix-poetry-cache-test
    • 22c6af9 Change PyPy version to rebuild cache
    • 081a3cf Merge pull request #405 from mayeut/interpreter-path
    • ff70656 feature: add a python-path output
    • fff15a2 Use pypyX.Y for PyPy python-version input (#349)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/cache from 3.0.2 to 3.0.4

    Bump actions/cache from 3.0.2 to 3.0.4

    Bumps actions/cache from 3.0.2 to 3.0.4.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.4

    In this release, we have fixed the tar creation error while trying to create it with path as ~/ home folder on ubuntu-latest.

    v3.0.3

    Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

    Changelog

    Sourced from actions/cache's changelog.

    3.0.2

    • Added support for dynamic cache size cap on GHES.

    3.0.3

    • Fixed avoiding empty cache save when no files are available for caching. (issue)

    3.0.4

    • Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)
    Commits
    • c3f1317 Merge pull request #813 from actions/users/kotewar/upgrading-cache-to-v2.0.6
    • d0a54b9 Fixed typo
    • 8c5bd0c Updated README file with release info
    • c9c0f73 Merge pull request #812 from actions/users/kotewar/upgrading-cache-to-v2.0.6
    • 2b6caae Merge pull request #495 from ostera/patch-1
    • dd58d13 Added release info and upgraded version
    • acace7f Merge branch 'main' into patch-1
    • 438628a Merge pull request #554 from albertstill/improve-restore-key-docs
    • c296e6a Updated @​actions/cache version in license file
    • 7ed7f22 Updated actions/cache to v2.0.6
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump actions/cache from 3.0.2 to 3.0.3

    Bump actions/cache from 3.0.2 to 3.0.3

    Bumps actions/cache from 3.0.2 to 3.0.3.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.3

    Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

    Changelog

    Sourced from actions/cache's changelog.

    3.0.2

    • Added support for dynamic cache size cap on GHES.

    3.0.3

    • Fixed avoiding empty cache save when no files are available for caching. (issue)
    Commits
    • 30f413b Merge pull request #808 from actions/users/kotewar/update-actions-cache-version
    • f2ea0f2 Updated Readme with what's new and removed v2 info
    • d5956bb Updated release and package versions
    • a686c72 Revert "Updated package version to 3.0.3 and dependencies"
    • 8e680ea Revert "Added release information for 3.0.2 in RELEASES.md"
    • c376fc8 Added release information for 3.0.2 in RELEASES.md
    • 4ce1a96 Updated package version to 3.0.3 and dependencies
    • a2bd439 Updated license version and dist
    • 35ef551 Upgraded @​actions/cache to 2.0.5
    • 14c4fd4 Merge pull request #690 from pimterry/patch-1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump codecov/codecov-action from 2.1.0 to 3.0.0

    Bump codecov/codecov-action from 2.1.0 to 3.0.0

    Bumps codecov/codecov-action from 2.1.0 to 3.0.0.

    Release notes

    Sourced from codecov/codecov-action's releases.

    v3.0.0

    Breaking Changes

    • #689 Bump to node16 and small fixes

    Features

    • #688 Incorporate gcov arguments for the Codecov uploader

    Dependencies

    • #548 build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0
    • #603 [Snyk] Upgrade @​actions/core from 1.5.0 to 1.6.0
    • #628 build(deps): bump node-fetch from 2.6.1 to 3.1.1
    • #634 build(deps): bump node-fetch from 3.1.1 to 3.2.0
    • #636 build(deps): bump openpgp from 5.0.1 to 5.1.0
    • #652 build(deps-dev): bump @​vercel/ncc from 0.30.0 to 0.33.3
    • #653 build(deps-dev): bump @​types/node from 16.11.21 to 17.0.18
    • #659 build(deps-dev): bump @​types/jest from 27.4.0 to 27.4.1
    • #667 build(deps): bump actions/checkout from 2 to 3
    • #673 build(deps): bump node-fetch from 3.2.0 to 3.2.3
    • #683 build(deps): bump minimist from 1.2.5 to 1.2.6
    • #685 build(deps): bump @​actions/github from 5.0.0 to 5.0.1
    • #681 build(deps-dev): bump @​types/node from 17.0.18 to 17.0.23
    • #682 build(deps-dev): bump typescript from 4.5.5 to 4.6.3
    • #676 build(deps): bump @​actions/exec from 1.1.0 to 1.1.1
    • #675 build(deps): bump openpgp from 5.1.0 to 5.2.1
    Changelog

    Sourced from codecov/codecov-action's changelog.

    3.0.0

    Breaking Changes

    • #689 Bump to node16 and small fixes

    Features

    • #688 Incorporate gcov arguments for the Codecov uploader

    Dependencies

    • #548 build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0
    • #603 [Snyk] Upgrade @​actions/core from 1.5.0 to 1.6.0
    • #628 build(deps): bump node-fetch from 2.6.1 to 3.1.1
    • #634 build(deps): bump node-fetch from 3.1.1 to 3.2.0
    • #636 build(deps): bump openpgp from 5.0.1 to 5.1.0
    • #652 build(deps-dev): bump @​vercel/ncc from 0.30.0 to 0.33.3
    • #653 build(deps-dev): bump @​types/node from 16.11.21 to 17.0.18
    • #659 build(deps-dev): bump @​types/jest from 27.4.0 to 27.4.1
    • #667 build(deps): bump actions/checkout from 2 to 3
    • #673 build(deps): bump node-fetch from 3.2.0 to 3.2.3
    • #683 build(deps): bump minimist from 1.2.5 to 1.2.6
    • #685 build(deps): bump @​actions/github from 5.0.0 to 5.0.1
    • #681 build(deps-dev): bump @​types/node from 17.0.18 to 17.0.23
    • #682 build(deps-dev): bump typescript from 4.5.5 to 4.6.3
    • #676 build(deps): bump @​actions/exec from 1.1.0 to 1.1.1
    • #675 build(deps): bump openpgp from 5.1.0 to 5.2.1
    Commits
    • e3c5604 Merge pull request #689 from codecov/feat/gcov
    • 174efc5 Update package-lock.json
    • 6243a75 bump to 3.0.0
    • 0d6466f Bump to node16
    • d4729ee fetch.default
    • 351baf6 fix: bash
    • d8cf680 Merge pull request #675 from codecov/dependabot/npm_and_yarn/openpgp-5.2.1
    • b775e90 Merge pull request #676 from codecov/dependabot/npm_and_yarn/actions/exec-1.1.1
    • 2ebc2f0 Merge pull request #682 from codecov/dependabot/npm_and_yarn/typescript-4.6.3
    • 8e2ef2b Merge pull request #681 from codecov/dependabot/npm_and_yarn/types/node-17.0.23
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Re-add numpy nightlies

    Re-add numpy nightlies

    Currently the nightly tests with numpy are failing since a significant period with np.array([[np.nan]]).astype(str) leading to an error. We should investigate the underlying issue and re-add the tests once we fixed those.

    opened by xhochy 0
  • Remove `:okwarning:` from documentation

    Remove `:okwarning:` from documentation

    In the process of fixing the documentation in #5 , I've added :okwarning: markers at various places. We should fix the warnings and remove the markers again then.

    opened by xhochy 0
Releases(4.1.4)
  • 4.1.4(Dec 14, 2022)

    What's Changed

    • Use pip for readthedocs installation by @xhochy in https://github.com/data-engineering-collective/plateau/pull/59
    • Bump mamba-org/provision-with-micromamba from 13 to 14 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/60
    • Update pre-commit hooks by @xhochy in https://github.com/data-engineering-collective/plateau/pull/63
    • Test for pyarrow=10 and Python=3.11 by @xhochy in https://github.com/data-engineering-collective/plateau/pull/69
    • Add code scanning with CodeQL by @xhochy in https://github.com/data-engineering-collective/plateau/pull/62
    • Don't package tests by @jtilly in https://github.com/data-engineering-collective/plateau/pull/61
    • Create CODEOWNERS by @xhochy in https://github.com/data-engineering-collective/plateau/pull/70
    • Update pre-commit hook to numpy=1.23 by @xhochy in https://github.com/data-engineering-collective/plateau/pull/71

    Full Changelog: https://github.com/data-engineering-collective/plateau/compare/4.1.3...4.1.4

    Source code(tar.gz)
    Source code(zip)
  • 4.1.3(Oct 24, 2022)

    What's Changed

    • Patch to load partition with only nulls as a category by @DamianBarabonkovQC in https://github.com/data-engineering-collective/plateau/pull/56

    Full Changelog: https://github.com/data-engineering-collective/plateau/compare/4.1.2...4.1.3

    Source code(tar.gz)
    Source code(zip)
  • 4.1.2(Oct 20, 2022)

    What's Changed

    • Remove pyarrow upper-bound version pin by @DamianBarabonkovQC in https://github.com/data-engineering-collective/plateau/pull/58

    Full Changelog: https://github.com/data-engineering-collective/plateau/compare/4.1.1...4.1.2

    Source code(tar.gz)
    Source code(zip)
  • 4.1.1(Oct 19, 2022)

    What's Changed

    • Fix usage of setuptools_scm after migrating to declarative build by @xhochy in https://github.com/data-engineering-collective/plateau/pull/57

    Full Changelog: https://github.com/data-engineering-collective/plateau/compare/4.1.0...4.1.1

    Source code(tar.gz)
    Source code(zip)
  • 4.1.0(Oct 19, 2022)

    What's Changed

    • Bump actions/cache from 2.1.7 to 3.0.2 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/31
    • Bump codecov/codecov-action from 2.1.0 to 3.1.0 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/32
    • Bump actions/cache from 3.0.2 to 3.0.8 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/41
    • Bump pre-commit/action from 2.0.3 to 3.0.0 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/34
    • Bump actions/setup-python from 3 to 4.1.0 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/37
    • Support pyarrow=6.* by @xhochy in https://github.com/data-engineering-collective/plateau/pull/42
    • Support numpy 1.23, pandas 1.4.{3,4}, pyarrow=4.0.1 by @xhochy in https://github.com/data-engineering-collective/plateau/pull/43
    • Support pyarrow={7,8,9}, dask>=2022.4.2; update mypy by @xhochy in https://github.com/data-engineering-collective/plateau/pull/44
    • Bump codecov/codecov-action from 3.1.0 to 3.1.1 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/46
    • Bump actions/cache from 3.0.8 to 3.0.10 by @dependabot in https://github.com/data-engineering-collective/plateau/pull/48
    • Update packaging setup by @xhochy in https://github.com/data-engineering-collective/plateau/pull/49
    • Check untyped defs by @xhochy in https://github.com/data-engineering-collective/plateau/pull/50
    • Replace storefact and simplekv usages with minimalkv by @xhochy in https://github.com/data-engineering-collective/plateau/pull/51
    • Test for pyarrow 5 by @xhochy in https://github.com/data-engineering-collective/plateau/pull/52
    • Update pre-commit hooks and add pyupgrade by @xhochy in https://github.com/data-engineering-collective/plateau/pull/53
    • Use mamba for readthedocs. by @jtilly in https://github.com/data-engineering-collective/plateau/pull/54

    New Contributors

    • @jtilly made their first contribution in https://github.com/data-engineering-collective/plateau/pull/54

    Full Changelog: https://github.com/data-engineering-collective/plateau/compare/4.0.4...4.1.0

    Source code(tar.gz)
    Source code(zip)
  • 4.0.4(Mar 17, 2022)

    Plateau 4.0.4

    • Fork and rename the project as plateau (flat files, flat land).
    • Fixed a bug where sometimes data partitions in a plateau dataset would be read in a non-deterministic order among Python executions (#23).
    Source code(tar.gz)
    Source code(zip)
Owner
null
dotsend is a web application which helps you to upload your large files and share file via link

dotsend is a web application which helps you to upload your large files and share file via link

Devocoe 0 Dec 3, 2022
A tool for batch processing large fasta files and accompanying metadata table to upload to repositories via API

Fasta Uploader A tool for batch processing large fasta files and accompanying metadata table to repositories via API The python fasta_uploader.py scri

Centre for Infectious Disease and One Health 1 Dec 9, 2021
Python library and shell utilities to monitor filesystem events.

Watchdog Python API and shell utilities to monitor file system events. Works on 3.6+. If you want to use Python 2.6, you should stick with watchdog <

Yesudeep Mangalapilly 5.6k Jan 4, 2023
Powerful Python library for atomic file writes.

Powerful Python library for atomic file writes.

Markus Unterwaditzer 313 Oct 19, 2022
ValveVMF - A python library to parse Valve's VMF files

ValveVMF ValveVMF is a Python library for parsing .vmf files for the Source Engi

pySourceSDK 2 Jan 2, 2022
Uproot is a library for reading and writing ROOT files in pure Python and NumPy.

Uproot is a library for reading and writing ROOT files in pure Python and NumPy. Unlike the standard C++ ROOT implementation, Uproot is only an I/O li

Scikit-HEP Project 164 Dec 31, 2022
A Python library that provides basic functions to read / write Aseprite format files

A Python library that provides basic functions to read / write Aseprite format files

Joe Trewin 1 Jan 13, 2022
A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".

the problem What directory should your app use for storing user data? If running on macOS, you should use: ~/Library/Application Support/<AppName> If

ActiveState Software 948 Dec 31, 2022
This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it

This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it. In the current state, it outputs a PrettyTable to txt file as well as the raw data from that table into a csv.

Joshua Wren 1 Nov 9, 2021
Fast Python reader and editor for ASAM MDF / MF4 (Measurement Data Format) files

asammdf is a fast parser and editor for ASAM (Association for Standardization of Automation and Measuring Systems) MDF (Measurement Data Format) files

Daniel Hrisca 440 Dec 31, 2022
shred - A cross-platform library for securely deleting files beyond recovery.

shred Help the project financially: Donate: https://smartlegion.github.io/donate/ Yandex Money: https://yoomoney.ru/to/4100115206129186 PayPal: https:

null 4 Sep 4, 2021
An easy-to-use library for emulating code in minidump files.

dumpulator Note: This is a work-in-progress prototype, please treat it as such. An easy-to-use library for emulating code in minidump files. Example T

Duncan Ogilvie 362 Dec 31, 2022
Simple, convenient and cross-platform file date changing library. 📝📅

Simple, convenient and cross-platform file date changing library.

kubinka0505 15 Dec 18, 2022
A simple library for temporary storage of small files

TemporaryStorage An simple library for temporary storage of small files. Navigation Install Usage In Python console As a standalone application List o

null 2 Apr 17, 2022
Sheet Data Image/PDF-to-CSV Converter

Sheet Data Image/PDF-to-CSV Converter

Quy Truong 5 Nov 22, 2021
Search for files under the specified directory. Extract the file name and file path and import them as data.

Search for files under the specified directory. Extract the file name and file path and import them as data. Based on that, search for the file, select it and open it.

G-jon FujiYama 2 Jan 10, 2022
Python Fstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems.

PyFstab Generator PyFstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems. NOTE : Th

Mahdi 2 Nov 9, 2021
Python's Filesystem abstraction layer

PyFilesystem2 Python's Filesystem abstraction layer. Documentation Wiki API Documentation GitHub Repository Blog Introduction Think of PyFilesystem's

pyFilesystem 1.8k Jan 2, 2023
A python wrapper for libmagic

python-magic python-magic is a Python interface to the libmagic file type identification library. libmagic identifies file types by checking their hea

Adam Hupp 2.3k Dec 29, 2022