fds is a tool for Data Scientists made by DAGsHub to version control data and code at once.

Overview

Fast Data Science aka fds

Discord Tests PyPI MIT license


fds is a tool for Data Scientists made by DAGsHub to version control data and code at once.

At a high level, fds is a command line wrapper around Git and DVC, meant to minimize the chances of human error, automate repetitive tasks, and provide a smoother landing for new users.

See the launch blog for more information about the motivation behind this project.

Installation

  • Install fds using PIP pip install fastds
  • Once installed successfully, you can start using fds
  • eg: fds init should trigger the init command
  • You can also use sdf instead of fds - it's identical, but might be more fun to type 🤓

Commands Supported

$ fds -h
usage: fds [-h] [-v] {init,status,add,commit,push,save} ...

One command for all your git and dvc needs

positional arguments:
  {init,status,add,commit,push,save}
                        command (refer commands section in documentation)
    init                initialize a git and dvc repository
    status              get status of your git and dvc repository
    add                 add files/folders to git and dvc repository
    commit              commits added changes to git and dvc repository
    push                push commits to remote git and dvc repository
    save                saves all project files to a new version and pushes
                        them to your remote

Examples

fds status = dvc status + git status

fds status lets us quickly check the full status of the repo - both DVC and git at the same time, to make sure we don't forget anything.

image

Here, we can see that we have a small, normal text file - .gitignore, plus a bigfile.txt and data folder which we would want to add to DVC and not to git. fds add makes that easy!

fds add = dvc add + git add wizard 🧙‍♂️

You're probably used to the convenience of using git add . to just track everything. Unfortunately, you have to be careful doing this when working with large files - one wrong move, and you might fry your hard drive by accidentally telling git to track a huge dataset!
We wanted to retain the convenience of just typing one command which means "just track all changes, I'll do a git commit in one second", which will be smart enough to avoid the pitfalls of large data files.
fds add does exactly that, while interactively asking the user how to handle files. You can add to DVC, or git, recursively step into large folders, skip or ignore files, etc.

image

Here's the file tree of the repo I used above, with file sizes included. Note how bigfile.txt and data/ were automatically added to DVC and not git:

image

fds commit = dvc commit + git commit

Finally, to close the loop of a real workflow, what happens when I change existing DVC tracked files? Without FDS, you'd have to remember to separately run dvc repro or dvc commit, then git add tracked_file.dvc, and only then git commit.
fds commit does all that for you - commits changes to DVC first, then adds the .dvc files with the updated hashes to git, then immediately commits these changes (plus any other staged changes) to a new git commit. Voila!

image

Contributing

We would love for you to try out FDS yourself, and to give us feedback. It would really help us to prioritize future features, so please vote on or create issues!
If you'd like to take a more active part, we have some good first issues that you can start with. We'll be happy to provide guidance on the best way to do so.

And of course, we're always happy to have you on the DAGsHub discord, where you can ask questions or give feedback on FDS: Discord


Made with ❤️   by dagshub-logo
Comments
  • Check latest version on PyPI and suggest to upgrade

    Check latest version on PyPI and suggest to upgrade

    On any command of fds, it should check if a new version is available and either display a message clearly suggesting to upgrade, or actually ask whether to upgrade interactively. Not sure if the interactive upgrade is actually possible though.

    enhancement good first issue ux 
    opened by guysmoilov 10
  • fds init

    fds init

    I successfully pip installed fastds, but anytime I did fds init, it always gives this import output after displaying some Traceback information: ImportError: cannot import name '__version__' from 'fds.__init__' < c:\users\<MyComputerName>\pneumonia-detection\.venv\lib\site-packages\fds\__init__.py>

    bug 
    opened by Chibuzo75 9
  • Fail tests on flake8 failure

    Fail tests on flake8 failure

    For syntax errors we anyways fail the test, for the other errors we show them as warnings, but if the total count of these exceeds 10 then we throw an error asking user to fix those lint errors

    tests 
    opened by mohithg 5
  • Documentation request: using fds with git repo already using dvc

    Documentation request: using fds with git repo already using dvc

    It's not clear from the readme, or the blog posting, how to start using fds with a git repo already using dvc.

    • https://github.com/DAGsHub/fds/blob/main/README.md
    • https://dagshub.com/blog/fds-fast-data-science-with-git-and-dvc/

    I can guess that all I need to do is install the fds python package, and I use pipenv, so that might work, but it isn't clear. I know "make a copy of your repo and try it out!" but how many experiments until I recreate the documentation that is so badly needed?

    documentation 
    opened by drjasonharrison 4
  • feat(run): implement pre-command hook runner to handle edge cases

    feat(run): implement pre-command hook runner to handle edge cases

    • New HooksRunner cls to contain pre-command hooks
    • Move existing hooks+logic into new class
    • Automatically trigger (with user confirmation) git init / dvc init

    fixes #35

    enhancement 
    opened by pwoolvett 4
  • Feature to check for latest version before running a command

    Feature to check for latest version before running a command

    With the package version info from PyPi, fastds will now compare the local version with the latest version available on PyPI, to prompt for an upgrade if the local version is not the latest.

    enhancement ux 
    opened by indweller 4
  • fds clone need extra argument folder location

    fds clone need extra argument folder location

    For Git you can simply set the location of local repository. For example

    git clone <remote> <local directory>
    
    

    It is not urgent issue, but in annoyance category.

    ux 
    opened by kingabzpro 2
  • Auto update FDS only tries with `pip` and not `pip3`

    Auto update FDS only tries with `pip` and not `pip3`

    fds asks if I want to update to the latest version, but when I type y to perform the update, I get thrown a bad interpreter error, due to it using pip which doesn't work on my system for a few reasons.

    It would be great if fds could check whether pip or pip3 work for the update, and use whichever one works.

    enhancement 
    opened by deanp70 2
  • Add `-y` option to let user decide to auto confirm all changed files

    Add `-y` option to let user decide to auto confirm all changed files

    As a side note, I think the way I added the argument makes more sense compared to how the --verbose flag was added, since it would currently accept an argument, which might cause issues. I haven't checked this, but @mohithg or @guysmoilov let me know if I'm missing something or we should change it too.

    enhancement 
    opened by deanp70 2
  • fds not working on Mac with whitespaces in the path

    fds not working on Mac with whitespaces in the path

    The path to my venv is /Users/deanpleban/Project Talos/Code/fds/env/bin/fds and whenever I run any fds command I get:

    /Users/deanpleban/Project Talos/Code/fds/env/bin/fds: bad interpreter: "/Users/deanpleban/Project: no such file or directory
    

    I tried googling around, but so far I can't find an explanation for how to solve this.

    opened by deanp70 2
  • Update pytest requirement from ~=6.2.5 to ~=7.1.3

    Update pytest requirement from ~=6.2.5 to ~=7.1.3

    Updates the requirements on pytest to permit the latest version.

    Release notes

    Sourced from pytest's releases.

    7.1.3

    pytest 7.1.3 (2022-08-31)

    Bug Fixes

    • #10060: When running with --pdb, TestCase.tearDown is no longer called for tests when the class has been skipped via unittest.skip or pytest.mark.skip.
    • #10190: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.
    • #10230: Ignore .py files created by pyproject.toml-based editable builds introduced in pip 21.3.
    • #3396: Doctests now respect the --import-mode flag.
    • #9514: Type-annotate FixtureRequest.param as Any as a stop gap measure until 8073{.interpreted-text role="issue"} is fixed.
    • #9791: Fixed a path handling code in rewrite.py that seems to work fine, but was incorrect and fails in some systems.
    • #9917: Fixed string representation for pytest.approx{.interpreted-text role="func"} when used to compare tuples.

    Improved Documentation

    • #9937: Explicit note that tmpdir{.interpreted-text role="fixture"} fixture is discouraged in favour of tmp_path{.interpreted-text role="fixture"}.

    Trivial/Internal Changes

    Commits
    • 4645bcd Remove incorrect output in how-to/fixtures.rst
    • fadfb4f Prepare release version 7.1.3
    • ab96ea8 Merge pull request #10258 from pytest-dev/backport-10252-to-7.1.x
    • fc0e024 [7.1.x] Fix regendoc
    • 8f5088f Merge pull request #10249 from pytest-dev/backport-10231-to-7.1.x
    • aae93d6 Ignore type-errors related to attr.asdict
    • 71b79fc [7.1.x] Ignore editable installation modules
    • 89f7518 Merge pull request #10222 from pytest-dev/backport-10171-to-7.1.x
    • 88fc45b [7.1.x] Update fixtures.rst w/ finalizer order
    • d0b53d6 Merge pull request #10221 from pytest-dev/backport-10217-to-7.1.x
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Update pytest requirement from ~=6.2.5 to ~=7.2.0

    Update pytest requirement from ~=6.2.5 to ~=7.2.0

    Updates the requirements on pytest to permit the latest version.

    Release notes

    Sourced from pytest's releases.

    7.2.0

    pytest 7.2.0 (2022-10-23)

    Deprecations

    • #10012: Update pytest.PytestUnhandledCoroutineWarning{.interpreted-text role="class"} to a deprecation; it will raise an error in pytest 8.

    • #10396: pytest no longer depends on the py library. pytest provides a vendored copy of py.error and py.path modules but will use the py library if it is installed. If you need other py.* modules, continue to install the deprecated py library separately, otherwise it can usually be removed as a dependency.

    • #4562: Deprecate configuring hook specs/impls using attributes/marks.

      Instead use :pypytest.hookimpl{.interpreted-text role="func"} and :pypytest.hookspec{.interpreted-text role="func"}. For more details, see the docs <legacy-path-hooks-deprecated>{.interpreted-text role="ref"}.

    • #9886: The functionality for running tests written for nose has been officially deprecated.

      This includes:

      • Plain setup and teardown functions and methods: this might catch users by surprise, as setup() and teardown() are not pytest idioms, but part of the nose support.
      • Setup/teardown using the @​with_setup decorator.

      For more details, consult the deprecation docs <nose-deprecation>{.interpreted-text role="ref"}.

    Features

    • #9897: Added shell-style wildcard support to testpaths.

    Improvements

    • #10218: @pytest.mark.parametrize() (and similar functions) now accepts any Sequence[str] for the argument names, instead of just list[str] and tuple[str, ...].

      (Note that str, which is itself a Sequence[str], is still treated as a comma-delimited name list, as before).

    • #10381: The --no-showlocals flag has been added. This can be passed directly to tests to override --showlocals declared through addopts.

    • #3426: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expresed instead.

    • #7337: A warning is now emitted if a test function returns something other than [None]{.title-ref}. This prevents a common mistake among beginners that expect that returning a [bool]{.title-ref} (for example [return foo(a, b) == result]{.title-ref}) would cause a test to pass or fail, instead of using [assert]{.title-ref}.

    • #8508: Introduce multiline display for warning matching via :pypytest.warns{.interpreted-text role="func"} and enhance match comparison for :py_pytest._code.ExceptionInfo.match{.interpreted-text role="func"} as returned by :pypytest.raises{.interpreted-text role="func"}.

    • #8646: Improve :pypytest.raises{.interpreted-text role="func"}. Previously passing an empty tuple would give a confusing error. We now raise immediately with a more helpful message.

    • #9741: On Python 3.11, use the standard library's tomllib{.interpreted-text role="mod"} to parse TOML.

      tomli{.interpreted-text role="mod"}` is no longer a dependency on Python 3.11.

    • #9742: Display assertion message without escaped newline characters with -vv.

    • #9823: Improved error message that is shown when no collector is found for a given file.

    ... (truncated)

    Commits
    • 3af3f56 Prepare release version 7.2.0
    • bc2c3b6 Merge pull request #10408 from NateMeyvis/patch-2
    • d84ed48 Merge pull request #10409 from pytest-dev/asottile-patch-1
    • ffe49ac Merge pull request #10396 from pytest-dev/pylib-hax
    • d352098 allow jobs to pass if codecov.io fails
    • c5c562b Fix typos in CONTRIBUTING.rst
    • d543a45 add deprecation changelog for py library vendoring
    • f341a5c Merge pull request #10407 from NateMeyvis/patch-1
    • 1027dc8 [pre-commit.ci] auto fixes from pre-commit.com hooks
    • 6b905ee Add note on tags to CONTRIBUTING.rst
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump colorama from 0.4.4 to 0.4.6

    Bump colorama from 0.4.4 to 0.4.6

    Bumps colorama from 0.4.4 to 0.4.6.

    Changelog

    Sourced from colorama's changelog.

    0.4.6 Current release

    • tartley/colorama#139 Add alternative to 'init()', called 'just_fix_windows_console'. This fixes many longstanding problems with 'init', such as working incorrectly on modern Windows terminals, and wonkiness when init gets called multiple times. The intention is that it just makes all Windows terminals treat ANSI the same way as other terminals do. Many thanks the njsmith for fixing our messes.
    • tartley/colorama#352 Support Windows 10's ANSI/VT console. This didn't exist when Colorama was created, and avoiding us causing havok there is long overdue. Thanks to segeviner for the initial approach, and to njsmith for getting it merged.
    • tartley/colorama#338 Internal overhaul of package metadata declaration, which abolishes our use of the now heavily discouraged setuptools (and hence setup.py, setup.cfg and MANIFEST.in), in favor of hatchling (and hence pyproject.toml), generously contributed by ofek (author of hatchling). This includes dropping support Python3.5 and 3.6, which are EOL, and were already dropped from setuptools, so this should not affect our users.
    • tartley/colorama#353 Attention to detail award to LqdBcnAtWork for a spelling fix in demo06 0.4.5
    • Catch a racy ValueError that could occur on exit.
    • Create README-hacking.md, for Colorama contributors.
    • Tweak some README unicode characters that don't render correctly on PyPI.
    • Fix some tests that were failing on some operating systems.
    • Add support for Python 3.9.
    • Add support for PyPy3.
    • Add support for pickling with the dill module.
    Commits
    • 3de9f01 bump version 0.4.6
    • a45949b Format the CHANGELOG bullet list, no content change
    • f55f72e comment need for a fix after recent MP broke it
    • cb83041 fix test-release after recent MPs broke it
    • 832f14c README tweaks
    • 54b89c3 Bump version to 0.4.6rc1
    • 7991d34 'make bootstrap' uses system python3, not pinned v3.8
    • 0ae5ef2 CHANGELOG updates for upcoming 0.4.6 release
    • 52f4cfd Tweak ordering of release checklist
    • ab64cfa Merge pull request #353 from LqdBcnAtWork/patch-1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump humanize from 3.13.1 to 4.4.0

    Bump humanize from 3.13.1 to 4.4.0

    Bumps humanize from 3.13.1 to 4.4.0.

    Release notes

    Sourced from humanize's releases.

    4.4.0

    Added

    Fixed

    4.3.0

    Added

    Fixed

    4.2.3

    Fixed

    • Update annotations, docs, and tests: naturaltime can also accept a timedelta (#31) @​nuztalgia

    4.2.2

    Fixed

    • Update annotations: naturadelta and naturaltime can also accept a float (#29) @​hugovk

    4.2.1

    Fixed

    4.2.0

    Added

    Fixed

    ... (truncated)

    Commits
    • 5553ff5 Update FUNDING.yml
    • 694a2d6 Merge pull request #56 from python-humanize/add-installation-instructions
    • c0591ae Add installation instructions
    • e0e82da Merge pull request #54 from mjmikulski/pl-short-scale
    • a52beed Add author notice
    • 7b0d9be Fix wrong declinations of googol and a year
    • d0dde8f Replace short scale with long scale for Polish
    • 72d3ef4 Merge pull request #53 from Luflosi/implement-decimal-sep-i18n
    • 862748a Internationalise the decimal separator in intcomma()
    • d8e2739 Merge pull request #52 from Luflosi/fix-intcomma-with-str-and-ndigits
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump requests from 2.26.0 to 2.28.1

    Bump requests from 2.26.0 to 2.28.1

    Bumps requests from 2.26.0 to 2.28.1.

    Release notes

    Sourced from requests's releases.

    v2.28.1

    2.28.1 (2022-06-29)

    Improvements

    • Speed optimization in iter_content with transition to yield from. (#6170)

    Dependencies

    • Added support for chardet 5.0.0 (#6179)
    • Added support for charset-normalizer 2.1.0 (#6169)

    New Contributors

    Full Changelog: https://github.com/psf/requests/blob/main/HISTORY.md#2281-2022-06-29

    v2.28.0

    2.28.0 (2022-06-09)

    Deprecations

    • ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ (#6091)
    • Requests has officially dropped support for Python 3.6 (including pypy3). (#6091)

    Improvements

    • Wrap JSON parsing issues in Request's JSONDecodeError for payloads without an encoding to make json() API consistent. (#6097)
    • Parse header components consistently, raising an InvalidHeader error in all invalid cases. (#6154)
    • Added provisional 3.11 support with current beta build. (#6155)
    • Requests got a makeover and we decided to paint it black. (#6095)

    Bugfixes

    • Fixed bug where setting CURL_CA_BUNDLE to an empty string would disable cert verification. All Requests 2.x versions before 2.28.0 are affected. (#6074)
    • Fixed urllib3 exception leak, wrapping urllib3.exceptions.SSLError with requests.exceptions.SSLError for content and iter_content. (#6057)
    • Fixed issue where invalid Windows registry entires caused proxy resolution to raise an exception rather than ignoring the entry. (#6149)
    • Fixed issue where entire payload could be included in the error message for JSONDecodeError. (#6079)

    New Contributors

    ... (truncated)

    Changelog

    Sourced from requests's changelog.

    2.28.1 (2022-06-29)

    Improvements

    • Speed optimization in iter_content with transition to yield from. (#6170)

    Dependencies

    • Added support for chardet 5.0.0 (#6179)
    • Added support for charset-normalizer 2.1.0 (#6169)

    2.28.0 (2022-06-09)

    Deprecations

    • ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ (#6091)
    • Requests has officially dropped support for Python 3.6 (including pypy3.6). (#6091)

    Improvements

    • Wrap JSON parsing issues in Request's JSONDecodeError for payloads without an encoding to make json() API consistent. (#6097)
    • Parse header components consistently, raising an InvalidHeader error in all invalid cases. (#6154)
    • Added provisional 3.11 support with current beta build. (#6155)
    • Requests got a makeover and we decided to paint it black. (#6095)

    Bugfixes

    • Fixed bug where setting CURL_CA_BUNDLE to an empty string would disable cert verification. All Requests 2.x versions before 2.28.0 are affected. (#6074)
    • Fixed urllib3 exception leak, wrapping urllib3.exceptions.SSLError with requests.exceptions.SSLError for content and iter_content. (#6057)
    • Fixed issue where invalid Windows registry entires caused proxy resolution to raise an exception rather than ignoring the entry. (#6149)
    • Fixed issue where entire payload could be included in the error message for JSONDecodeError. (#6036)

    2.27.1 (2022-01-05)

    Bugfixes

    • Fixed parsing issue that resulted in the auth component being dropped from proxy URLs. (#6028)

    2.27.0 (2022-01-03)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump validators from 0.18.2 to 0.20.0

    Bump validators from 0.18.2 to 0.20.0

    Bumps validators from 0.18.2 to 0.20.0.

    Changelog

    Sourced from validators's changelog.

    0.20.0 (2022-06-05) ^^^^^^^^^^^^^^^^^^^

    • Added ipv4 digit lenghts validation (#191, pull request courtesy of Norbiox)
    • Fixes error with international URLs that have more than 2 hyphens (#184, pull request courtesy of automationator)

    0.19.0 (2022-05-04) ^^^^^^^^^^^^^^^^^^^

    • Dropped py34 support
    • Improve IPv6 validation (#201, pull request courtesy of SimonIT)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • fds fails to pull dvc on windows

    fds fails to pull dvc on windows

    When running pull command on DagsHub remote. I receive dvc pull failure, so I have to manually pull dvc again.

    This issue permanent issue on windows.

    fds clone <remote> 
    
    

    It is not urgent issue, but in annoyance category.

    bug 
    opened by kingabzpro 3
Releases(v0.6.0)
  • v0.6.0(Nov 7, 2021)

  • v0.5.2(Oct 27, 2021)

  • v0.5.1(Oct 17, 2021)

    Changes

    🐛 Bug Fixes

    • Make hooks work as per commands @mohithg (#99)

    📦 Dependency updates

    • Bump humanize from 3.11.0 to 3.12.0 @dependabot (#103)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Sep 23, 2021)

    Changes

    🐛 Bug Fixes

    • Move __version__ to a version file @mohithg (#98)
    • Use dvc version 2.3.0 across the CLI @mohithg (#100)

    📦 Dependency updates

    • Use dvc version 2.3.0 across the CLI @mohithg (#100)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Sep 2, 2021)

    Changes

    🚀 Features

    • Make pip3 as default @mohithg (#96)
    • Fix dvc clone to ask if its git only project @mohithg (#88)

    🐛 Bug Fixes

    • Fix dvc clone to ask if its git only project @mohithg (#88)

    📦 Dependency updates

    • Update pytest requirement from ~=6.2.4 to ~=6.2.5 @dependabot (#95)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Aug 29, 2021)

    Changes

    🚀 Features

    • feat(run): implement pre-command hook runner to handle edge cases @pwoolvett (#91)

    🐛 Bug Fixes

    • Detect root of git and dvc @mohithg (#93)

    📝 Documentation updates

    • Fix documentation for fds clone @mohithg (#90)

    💥 Breaking changes

    • Detect root of git and dvc @mohithg (#93)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Aug 14, 2021)

  • v0.3.1v2(Aug 5, 2021)

    Changes

    🚀 Features

    • Integrate dvc auth with dvc pull @mohithg (#83)

    📦 Dependency updates

    • Bump progress from 1.5 to 1.6 @dependabot (#81)
    • Bump humanize from 3.10.0 to 3.11.0 @dependabot (#84)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jul 29, 2021)

    Changes

    🚀 Features

    • Make fds push ask for credentials of user if its not authenticated already @mohithg (#80)

    🎨 UX improvements

    • Make fds push ask for credentials of user if its not authenticated already @mohithg (#80)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Jul 19, 2021)

  • v0.2.1(Jul 15, 2021)

    Changes

    🚀 Features

    • Add skip option to fds add wizard @mohithg (#74)

    🐛 Bug Fixes

    • Add ignored files to dvcignore too @mohithg (#75)

    📦 Dependency updates

    • Bump requests from 2.25.1 to 2.26.0 @dependabot (#77)
    • Bump humanize from 3.9.0 to 3.10.0 @dependabot (#72)

    🎨 UX improvements

    • Add skip option to fds add wizard @mohithg (#74)
    • Add ignored files to dvcignore too @mohithg (#75)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jul 8, 2021)

    Changes

    🚀 Features

    • Feature to check for latest version before running a command @indweller (#73)
    • Implement clone command @mohithg (#53)

    🎨 UX improvements

    • Feature to check for latest version before running a command @indweller (#73)
    • Show an adding message before showing wizard in DVC add @mohithg (#70)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.8(Jun 29, 2021)

    Changes

    📦 Dependency updates

    • Bump humanize from 3.8.0 to 3.9.0 @dependabot (#59)

    🚦 Tests

    • Fail tests on flake8 failure @mohithg (#63)

    🚀 Performance

    • Improve fds add performance by checking dvc-ignore when necessary @mohithg (#67)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.7(Jun 17, 2021)

  • v0.1.6(Jun 14, 2021)

  • v0.1.5(Jun 14, 2021)

  • v0.1.4(Jun 7, 2021)

    Changes

    🐛 Bug Fixes

    • Add Integration tests and fix a bug with fds add <large file> that doesn't add into dvc @mohithg (#50)

    📦 Dependency updates

    • Bump humanize from 3.7.0 to 3.7.1 @dependabot (#51)
    • Bump humanize from 3.6.0 to 3.7.0 @dependabot (#49)
    • Bump pygit2 from 1.5.0 to 1.6.0 @dependabot (#48)

    🚦 Tests

    • Add Integration tests and fix a bug with fds add <large file> that doesn't add into dvc @mohithg (#50)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.3(Jun 2, 2021)

    Changes

    📦 Dependency updates

    • Bump humanize from 3.5.0 to 3.6.0 @dependabot (#45)

    🎨 UX improvements

    • Improve UX by showing progress bar while adding large files to DVC and other minor fixes to improve usability @mohithg (#46)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(May 27, 2021)

  • v0.1.1(May 27, 2021)

  • v0.1.0(May 26, 2021)

    Changes

    • Fix tests for fds @mohithg (#30)
    • Install DVC @mohithg (#8)
    • Implement commit command @mohithg (#7)
    • Dont track files/folders ignored in DVC and Git @mohithg (#6)
    • Fix feedback @mohithg (#5)
    • Remove dvc as a requirement for fds @mohithg (#4)
    • Add command @mohithg (#3)
    • Rename ff to fds @mohithg (#1)
    • Guy's feedback implementation @mohithg (#2)

    🚀 Features

    • Add -y option to let user decide to auto confirm all changed files @deanp70 (#26)
    • fds commands will now try to resume after installing dvc @guysmoilov (#22)

    🐛 Bug Fixes

    • Stop execution if command throws some error @mohithg (#28)
    • Removed non-working markdown from dvc install prompt @guysmoilov (#25)
    • Fix fds commit bug when there are actual changes @deanp70 (#12)

    🧰 Maintenance

    • Github action to upload to PyPI on GH release @guysmoilov (#33)
    • Github actions for python linting and testing @guysmoilov (#32)

    📝 Documentation updates

    • Update README.md @guysmoilov (#29)

    📦 Dependency updates

    • Update pytest requirement from ~=6.2.2 to ~=6.2.4 @dependabot (#23)
    • Bump humanize from 3.4.1 to 3.5.0 @dependabot (#24)

    🚦 Tests

    • Github actions for python linting and testing @guysmoilov (#32)
    Source code(tar.gz)
    Source code(zip)
Owner
DAGsHub
The home for data science collaboration
DAGsHub
VevestaX is an open source Python package for ML Engineers and Data Scientists.

VevestaX Track failed and successful experiments as well as features. VevestaX is an open source Python package for ML Engineers and Data Scientists.

Vevesta 24 Dec 14, 2022
:truck: Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

To launch a live notebook server to test optimus using binder or Colab, click on one of the following badges: Optimus is the missing framework to prof

Iron 1.3k Dec 30, 2022
This tool parses log data and allows to define analysis pipelines for anomaly detection.

logdata-anomaly-miner This tool parses log data and allows to define analysis pipelines for anomaly detection. It was designed to run the analysis wit

AECID 32 Nov 27, 2022
CaterApp is a cross platform, remotely data sharing tool created for sharing files in a quick and secured manner.

CaterApp is a cross platform, remotely data sharing tool created for sharing files in a quick and secured manner. It is aimed to integrate this tool with several more features including providing a User Interface.

Ravi Prakash 3 Jun 27, 2021
Driver Analysis with Factors and Forests: An Automated Data Science Tool using Python

Driver Analysis with Factors and Forests: An Automated Data Science Tool using Python ??

Thomas 2 May 26, 2022
Tuplex is a parallel big data processing framework that runs data science pipelines written in Python at the speed of compiled code

Tuplex is a parallel big data processing framework that runs data science pipelines written in Python at the speed of compiled code. Tuplex has similar Python APIs to Apache Spark or Dask, but rather than invoking the Python interpreter, Tuplex generates optimized LLVM bytecode for the given pipeline and input data set.

Tuplex 791 Jan 4, 2023
ForecastGA is a Python tool to forecast Google Analytics data using several popular time series models.

ForecastGA is a tool that combines a couple of popular libraries, Atspy and googleanalytics, with a few enhancements.

JR Oakes 36 Jan 3, 2023
Flenser is a simple, minimal, automated exploratory data analysis tool.

Flenser Have you ever been handed a dataset you've never seen before? Flenser is a simple, minimal, automated exploratory data analysis tool. It runs

John McCambridge 79 Sep 20, 2022
MoRecon - A tool for reconstructing missing frames in motion capture data.

MoRecon - A tool for reconstructing missing frames in motion capture data.

Yuki Nishidate 38 Dec 3, 2022
🧪 Panel-Chemistry - exploratory data analysis and build powerful data and viz tools within the domain of Chemistry using Python and HoloViz Panel.

???? ??. The purpose of the panel-chemistry project is to make it really easy for you to do DATA ANALYSIS and build powerful DATA AND VIZ APPLICATIONS within the domain of Chemistry using using Python and HoloViz Panel.

Marc Skov Madsen 97 Dec 8, 2022
Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

null 2 Nov 20, 2021
Elementary is an open-source data reliability framework for modern data teams. The first module of the framework is data lineage.

Data lineage made simple, reliable, and automated. Effortlessly track the flow of data, understand dependencies and analyze impact. Features Visualiza

null 898 Jan 9, 2023
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

Coiled 102 Nov 10, 2022
PrimaryBid - Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift

Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift This project is composed of two parts: Part1 and Part2

Emmanuel Boateng Sifah 1 Jan 19, 2022
This is a tool for speculation of ancestral allel, calculation of sfs and drawing its bar plot.

superSFS This is a tool for speculation of ancestral allel, calculation of sfs and drawing its bar plot. It is easy-to-use and runing fast. What you s

null 3 Dec 16, 2022
A tool to compare differences between dataframes and create a differences report in Excel

similarpanda A module to check for differences between pandas Dataframes, and generate a report in Excel format. This is helpful in a workplace settin

Andre Pretorius 9 Sep 15, 2022
Python data processing, analysis, visualization, and data operations

Python This is a Python data processing, analysis, visualization and data operations of the source code warehouse, book ISBN: 9787115527592 Descriptio

FangWei 1 Jan 16, 2022
Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

Data Scientist Learning Plan Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

Trung-Duy Nguyen 27 Nov 1, 2022
A neural-based binary analysis tool

A neural-based binary analysis tool Introduction This directory contains the demo of a neural-based binary analysis tool. We test the framework using

Facebook Research 208 Dec 22, 2022