CredSweeper is a tool to detect credentials in any directories or files.

Overview

CredSweeper

Table of Contents

Introduction

CredSweeper is a tool to detect credentials in any directories or files. CredSweeper could help users to detect unwanted exposure of credentials (such as personal information, token, passwords, api keys and etc) in advance. By scanning lines, filtering, and using AI model as option, CredSweeper reports lines with possible credentials, where the line is, and expected type of the credential as a result.

How To Use

Main Requirements

  • Python3.7 or higher

Installation

Via pip

Stable (release 1.0.0):

$ pip install git+https://github.com/Samsung/[email protected]

Latest (current master branch):

$ pip install git+https://github.com/Samsung/CredSweeper.git

Via git clone (dev install)

$ git clone https://github.com/Samsung/CredSweeper.git
$ cd CredSweeper
$ pip install -qr requirements.txt

Run

Get all argument list:

$ python -m credsweeper --help

usage: python -m credsweeper [-h] --path PATH [PATH ...] [--rules [PATH]] [--ml_validation] [--api_validation] [-j POSITIVE_INT] [--skip_ignored] [--save-json [PATH]] [--log LOG_LEVEL]

optional arguments:
  -h, --help            show this help message and exit
  --path PATH [PATH ...]
                        file or directory to scan
  --rules [PATH]        path of rule config file (default: credsweeper/rules/config.yaml)
  --ml_validation       ml validation option on
  --api_validation      api validation option on
  -j POSITIVE_INT, --jobs POSITIVE_INT
                        number of parallel processes to use (default: number of CPU cores * 2)
  --skip_ignored        parse .gitignore files and skip credentials from ignored objects
  --save-json [PATH]    save result to json file (default: output.json)
  --log [LOG_LEVEL]     set logging level. Example --log debug, (default: 'warning'), 
                          detailed log config: credsweeper/secret/log.yaml 

Get output as JSON file:

$ python -m credsweeper --ml_validation --path tests/samples/password --save-json output.json
rule: Password / severity: medium / line_data_list: [line : 'password = "cackle!"' / line_num : 1 / path : tests/samples/password / entropy_validation: False] / api_validation: NOT_AVAILABLE / ml_validation: VALIDATED_KEY
$ cat output.json
[
    {
        "rule": "Password",
        "severity": "medium",
        "line_data_list": [
            {
                "line": "password = \"cackle!\"",
                "line_num": 1,
                "path": "tests/samples/password",
                "entropy_validation": false
            }
        ],
        "api_validation": "NOT_AVAILABLE",
        "ml_validation": "VALIDATED_KEY"
    }
]

Get CLI output only:

$ python -m credsweeper --ml_validation --path tests/samples/password
rule: Password / severity: medium / line_data_list: [line : 'password = "cackle!"' / line_num : 1 / path : tests/samples/password / entropy_validation: False] / api_validation: NOT_AVAILABLE / ml_validation: VALIDATED_KEY

Tests

To run all tests:

$ python -m pytest --cov=credsweeper --cov-report=term-missing -s tests/

To run only tests independent from external api:

$ python -m pytest -m "not api_validation" --cov=credsweeper --cov-report=term-missing -s tests/

Benchmark

We have a dataset for testing credential scanners that called CredData. If you want to test CredSweeper with this dataset please check here.

Overall Architecture

To check overall architecture of CredSweeper please check here.

License

The CredSweeper is an Open Source project released under the terms of MIT License V2.

How to Get Involved

In addition to developing under an Open Source license, A use an Open Source Development approach, welcoming everyone to participate, contribute, and engage with each other through the project.

Project Roles

A recognizes the following formal roles: Contributor and Maintainer. Informally, the community may organize itself and give rights and responsibilities to the necessary people to achieve its goals.

Contributor

A Contributor is anyone who wishes to contribute to the project, at any level. Contributors are granted the following rights, to:

  • Contribute code, documentation, translations, artwork, and etc.
  • Report defects (bugs) and suggestions for enhancement.
  • Participate in the process of reviewing contributions by others.

Contributors are required to:

  • Receive the approval of the maintainer to contribute and reflect changes.
  • Assume responsibility for issues and bugs introduced by one's own contributions.
  • Provide constructive advice whenever participating in discussions and in the review of contributions.
  • Check code formatting before commit. The CredSweeper use yapf and to automate code formatting, run commands below.
    pip install pre-commit
    pre-commit install
    

Contributors who show dedication and skill are rewarded with additional rights and responsibilities. Their opinions weigh more when decisions are made, in a fully meritocratic fashion.

Maintainer

A Maintainer is a Contributor who is also responsible for knowing, directing and anticipating the needs of a given a Module. As such, Maintainers have the right to set the overall organization of the source code in the Module, and the right to participate in the decision-making. Maintainers are required to review the contributor’s requests and decide whether to accept or not.

Name E-Mail
Jaeku Yun [email protected]
Shinhyung Choi [email protected]
Yujeong Lee [email protected]
Oleksandra Sokol [email protected]

How to Contact

Please post questions, issues, or suggestions into Issues, This is the best way to communicate with the developer.

Comments
  • Filter out false positives

    Filter out false positives

    There is an option in config.json to exclude pattern and the README file includes a credential example. But this option in code is used to filter by file name, it does not filter credentials.

    There should be a way to filter out patterns that are known to be public keys or non-sensitive data of the project.

    enhancement 
    opened by rodrigost23 30
  • Add xml parser

    Add xml parser

    For now CredSweeper scan xml file line by line.

    So below case is detected now.

    <tool RefKey="DVmaxTimeTool" Name="EDT" />
    

    It is FP case and CredSweeper should focus on value not attribute. I want to make CredSweepr to detect below case.

    <?xml version='1.0' encoding='utf-8'?>
    <Countries>
        <Country name="Korea">
            <City attr1="value1" attr2="value2">Seoul</City>
            <password name="password">cackle!</password>
        </Country>
        <Country name="Ukraine">
            <City attr1="value1" attr2="value2">Kyiv</City>
            <password name="password">peace_for_ukraine</password>
        </Country>
    </Countries>
    

    And here is change

    • Add xml parser
    • Add and modify tests

    ~Actually there can be wrong line number on its result, because it is using xml parser so it is different with actual xml file.~

    opened by csh519 9
  • Benchmark workflow

    Benchmark workflow

    • Benchmark workflow for calculate scores. Using github cache reduces time to download CredData.
    • The workflow fails when scores are different. Need to update cicd/benchmark.txt.
    • Used v3 in all github actions
    • updated how-to-contribute
    • removed ActionTest from PR but keep in push trigger.
    enhancement 
    opened by babenek 7
  • Request for enhancement

    Request for enhancement

    I think that the algorithm of its work should be revised. For example, the utility reported 157 results. There are mostly false alarms:

    TEEC_ERROR_COMMUNICATIONPASSTEEC_ERROR_SHORT_BUFFERANY_OWNER_SETFAILInvoke_1param_valueInOuthexdump[%zu]:TEEC_InitializeContext
    Test passedERROR : TEEC_InvokeCommand failed: %s.
    Test passed.%s: OpenSession failed: %x.
    # Pass Cond   : TEE_SUCCESS
    # Pass Cond    : Passed if unit_tzld passes
    Test passedERROR : TEEC_InvokeCommand failed: %s.
     # Pass Cond    : Passed if average time.
    [swd_tzisr_test] TESTING : Thread 1:0x%08x:PASS:0x%08x
    [swd_tzisr_test] TESTING : 0x%08x:PASS:0x%08x
    cpuid[swd_tzisr_test] TESTING : Main Thread :0x%08x:PASS:0x%08x
    TEEC_ERROR_COMMUNICATIONPASSTEEC_ERROR_SHORT_BUFFERANY_OWNER_SETFAILhexdump[%zu]:TEEC_InitializeContext
    Test failedTest passedERROR: attack test failed: '%s' from '%s'.
    

    Such "creds" were found in a binary file. I haven't checked source codes of this repo, but there is a feeling that I will find there something like:

    strings BINARY_FILE | grep -i "pass"
    

    There have also been reports of this utility reporting "creds" found in the "DRM"ed (read encrypted) documents.

    Dear maintainers, please consider the cases of false positives in order to improve its performance.

    Thank you!

    opened by P1119r1m 7
  • Add Firebase rule

    Add Firebase rule

    Hello,

    Added Firebase detection rules.

    hack@hack:~$ cat strings.txt 
        <string name="firebase_database_url">https://api-project-615509201123.firebaseio.com</string>
        <string name="firebase_database_url">https://api-project-615509201123.firebaseapp.com</string>
    
    hack@hack:~$ python3 -m credsweeper --path /home/hack/strings.txt --rule /home/hack/config.yaml 
    rule: Firebase io Token / severity: high / line_data_list: [line : '    <string name="firebase_database_url">https://api-project-615509201123.firebaseio.com</string>' / line_num : 1 / path : /home/hack/strings.txt / value: 'api-project-615509201123.firebaseio.com' / entropy_validation: True] / api_validation: NOT_AVAILABLE / ml_validation: NOT_AVAILABLE
    rule: Firebase APP Token / severity: high / line_data_list: [line : '    <string name="firebase_database_url">https://api-project-615509201123.firebaseapp.com</string>' / line_num : 2 / path : /home/hack/strings.txt / value: 'api-project-615509201123.firebaseapp.com' / entropy_validation: True] / api_validation: NOT_AVAILABLE / ml_validation: NOT_AVAILABLE
    

    Thanks.

    Ref: https://hackerone.com/reports/1065134

    opened by gy741 6
  • Add option to exclude outputs with cli and config

    Add option to exclude outputs with cli and config

    Description

    Resolves #195

    • add blacklist field to the default config. Old configs without the fields will still be processed normally
    • add separate value and line blacklists. Both ignores space-like characters at left and right
    • add blacklist CLI arg that extends the list from config
    • at the moment CLI blacklist adds to both line and value blacklist for simplicity and user convenience. However, interface allows to add line and value lists separately
    • blacklist stored as Sets rather than Lists. Sets on average faster for x in y checks, and have average search time of O(1) https://wiki.python.org/moin/TimeComplexity
    • update the guide with blacklisting details

    image

    How has this been tested?

    Please describe the tests that you ran to verify your changes.

    • [x] Add new unit tests for both CLI arg and passing arguments directly into CredSweeper
    • [x] Verified proper changes in the guide
    enhancement 
    opened by meanrin 5
  • False positive for short regular expressions

    False positive for short regular expressions

    Many short regular expressions may randomly apear in the hashes, or base64 lines

    Example:

    https://github.com/tosca/web/blob/master/packages/EntityFramework.5.0.0/tools/EntityFramework.PS3.psd1#L172

    $ python -m credsweeper --path EntityFramework.PS3.psd1 
    rule: JSON Web Token / severity: medium / line_data_list: [line: '# UK3O3RhOJA/u0afRTK10MCAR6wfVVJUVSZQbQpKumFwwJtoAa+h7veyJBw/3DgSY' / line_num: 172 / path: EntityFramework.PS3.psd1 / value: 'eyJBw' / entropy_validation: False] / api_validation: NOT_AVAILABLE / ml_validation: VALIDATED_KEY
    

    99.9% sure it's not really a JWT

    While it is a JWT example, this can probably also happen with other short regular expressions (e.g. Google API based on ya29. prefix)

    As for now i see few options to solve it:

    1. Add prefix check and forbid matching regexes if they have a lot of alphanumeric stuff directly to the left. Example: =eyJBw... - ok, dasjkfseyJBw - bad
    2. Add minimal value length for some regexes. For example require minimal JWT length

    Maybe there are some different options. Please propose if you'll have any

    I haven't tested it with the CredData metrics yet, just ideas

    opened by meanrin 5
  • YAML and JSON are scanned like structures

    YAML and JSON are scanned like structures

    Description

    • Added StructContentProvider to separate structure storage
    • YAML and JSON might be scanned like structures
    • The case in yaml with multiline credential is found:
    key: |
      5UpEr5eCrEt
    

    How has this been tested?

    • [x] UnitTests
    • [x] do benchmark with --dept 33
    opened by babenek 3
  • Regarding release new version of CredSweeper.

    Regarding release new version of CredSweeper.

    Still we(@Samsung/credsweeper_maintainers ) just release new version of CredSweeper anytime we feel like it. I think it would be good to decide how to release new version and here is suggestions.

    Suggestions

    1. Anytime we feel like it. 2. Set period like 'once in a month'.

    Please let me know your opinions by comment.

    Thank you.

    enhancement 
    opened by csh519 3
  • Argument --depth and workaround

    Argument --depth and workaround

    The issue was created due #183 has to add second workaround to remove .gzextension from excluded list. First was .zip. The argument --depth enables file analysis not like plain text but with recursion if the file is an archive. Thus, class FilePathExtractor works before the analysis and removes true zip archives by extension. So, the feature might look like malfunction. Suppose, --depth will analyse various file types by content and the exclude list will be ignored. Simultaneously excluding by not_allowed_path_pattern and others are working before the list exluding. https://github.com/Samsung/CredSweeper/blob/eec642f420a3d27ff0e81489e1e01d61abb980bd/credsweeper/file_handler/file_path_extractor.py#L127-L135

    opened by babenek 3
  • Not working without error

    Not working without error

    I entered the following command, but I don't know if it worked.

    python -m credsweeper --path {MY_SRC_PATH} --save-json cred.json

    I opened the result file (cred.json), but there was only an empty array.

    I added some credential strings for test. But nothing has changed.

    Please help me.

    opened by securekim 3
Releases(v1.4.6)
  • v1.4.6(Dec 19, 2022)

    What's Changed

    • YAML and JSON are scanned like structures by @babenek in https://github.com/Samsung/CredSweeper/pull/236
    • Parse python sources to find full strings even they were split by @babenek in https://github.com/Samsung/CredSweeper/pull/237
    • Version up to 1.4.6 by @babenek in https://github.com/Samsung/CredSweeper/pull/244

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.5...v1.4.6

    Source code(tar.gz)
    Source code(zip)
  • v1.4.5(Nov 15, 2022)

    What's Changed

    • Deep scan enchancement by @babenek in https://github.com/Samsung/CredSweeper/pull/235
    • Extend help with possible variables by @babenek in https://github.com/Samsung/CredSweeper/pull/240
    • Propagate size limit config to PatchProvider by @babenek in https://github.com/Samsung/CredSweeper/pull/241
    • Version up to 1.4.5 by @babenek in https://github.com/Samsung/CredSweeper/pull/243
    • Used whathepatch v1.0.3 with fixed issue for huge diffs

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.4...v1.4.5

    Source code(tar.gz)
    Source code(zip)
  • v1.4.4(Oct 20, 2022)

    What's Changed

    • Add publickey filtering by @csh519 in https://github.com/Samsung/CredSweeper/pull/230
    • Added Info field to json report by @babenek in https://github.com/Samsung/CredSweeper/pull/225
    • File type is extracted to separated field for deep scan by @babenek in https://github.com/Samsung/CredSweeper/pull/232
    • Add ?, ! to key of KeywordPattern by @csh519 in https://github.com/Samsung/CredSweeper/pull/233

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.3...v1.4.4

    Source code(tar.gz)
    Source code(zip)
  • v1.4.3(Sep 22, 2022)

    What's Changed

    • Custom config for CredSweeper by @babenek in https://github.com/Samsung/CredSweeper/pull/191
    • Benchmark workflow by @babenek in https://github.com/Samsung/CredSweeper/pull/202
    • Add PR template by @csh519 in https://github.com/Samsung/CredSweeper/pull/201
    • Add option to exclude outputs with cli and config by @meanrin in https://github.com/Samsung/CredSweeper/pull/207
    • Benchmark workflow enchancement by @babenek in https://github.com/Samsung/CredSweeper/pull/209
    • Banner with version and CRC32 sum of program files by @babenek in https://github.com/Samsung/CredSweeper/pull/212
    • Modify benchmark workflow to use current code instead cloned by @babenek in https://github.com/Samsung/CredSweeper/pull/213
    • Updated scores after benchmark data updated by @babenek in https://github.com/Samsung/CredSweeper/pull/216
    • Argument --depth and workaround by @babenek in https://github.com/Samsung/CredSweeper/pull/215
    • Update KeywordPattern to more accuracy and cover "ENC(" case by @csh519 in https://github.com/Samsung/CredSweeper/pull/214
    • Pylint issue fix by @babenek in https://github.com/Samsung/CredSweeper/pull/218
    • Refactored unit test to avoid undefined behaviour by @babenek in https://github.com/Samsung/CredSweeper/pull/219
    • Custom log config by @babenek in https://github.com/Samsung/CredSweeper/pull/217
    • Version up 1.4.3 by @csh519 in https://github.com/Samsung/CredSweeper/pull/220

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.2...v1.4.3

    Source code(tar.gz)
    Source code(zip)
  • v1.4.2(Aug 29, 2022)

    What's Changed

    • Add config section to readme by @csh519 in https://github.com/Samsung/CredSweeper/pull/164
    • Auxiliary changes for tests by @babenek in https://github.com/Samsung/CredSweeper/pull/166
    • Catch more zipfile exceptions. by @babenek in https://github.com/Samsung/CredSweeper/pull/168
    • 2022 Aug 04 guide update by @meanrin in https://github.com/Samsung/CredSweeper/pull/173
    • Removed reference about char_to_index.pkl by @babenek in https://github.com/Samsung/CredSweeper/pull/171
    • Test help by @babenek in https://github.com/Samsung/CredSweeper/pull/174
    • Tests: fuzzing and optimizations by @babenek in https://github.com/Samsung/CredSweeper/pull/172
    • Move Tests and Benchmark section to Develop section by @csh519 in https://github.com/Samsung/CredSweeper/pull/176
    • Add completion phrase by @csh519 in https://github.com/Samsung/CredSweeper/pull/177
    • Mocking UT which use service with ban by @babenek in https://github.com/Samsung/CredSweeper/pull/181
    • Logging refactored by @babenek in https://github.com/Samsung/CredSweeper/pull/175
    • Update action.yml by @babenek in https://github.com/Samsung/CredSweeper/pull/185
    • Add minimal pattern value length by @meanrin in https://github.com/Samsung/CredSweeper/pull/182
    • Add .sum extension by @csh519 in https://github.com/Samsung/CredSweeper/pull/186
    • Left barrier for short patterns which may be found in long base64 data by @babenek in https://github.com/Samsung/CredSweeper/pull/184
    • Fuzz target refactored, corpus refuzzed by @babenek in https://github.com/Samsung/CredSweeper/pull/189
    • Raise exception log level to ERROR in api validation by @babenek in https://github.com/Samsung/CredSweeper/pull/188
    • UnitTests improvement by @babenek in https://github.com/Samsung/CredSweeper/pull/190
    • gzip format support by @babenek in https://github.com/Samsung/CredSweeper/pull/183
    • Consolidated json load-dump methods by @babenek in https://github.com/Samsung/CredSweeper/pull/187
    • Removed extra file by @babenek in https://github.com/Samsung/CredSweeper/pull/196
    • Update ml related documentation by @meanrin in https://github.com/Samsung/CredSweeper/pull/198
    • Add xml parser by @csh519 in https://github.com/Samsung/CredSweeper/pull/193
    • Update numpy version to 1.23.0 by @csh519 in https://github.com/Samsung/CredSweeper/pull/199
    • Version up 1.4.2 by @csh519 in https://github.com/Samsung/CredSweeper/pull/200

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.1...v1.4.2

    Source code(tar.gz)
    Source code(zip)
  • v1.4.1(Aug 2, 2022)

    What's Changed

    • Add --save-xlsx option by @csh519 in https://github.com/Samsung/CredSweeper/pull/156
    • Auxiliary refactoring by @babenek in https://github.com/Samsung/CredSweeper/pull/158
    • Fit encoding style between --save-json option and --save-xlsx option. by @csh519 in https://github.com/Samsung/CredSweeper/pull/159
    • Warning for huge lines by @babenek in https://github.com/Samsung/CredSweeper/pull/161
    • Add description of config.json and clean documents by @csh519 in https://github.com/Samsung/CredSweeper/pull/163
    • Fix and test AWS multi pattern by @babenek in https://github.com/Samsung/CredSweeper/pull/162
    • Update action.yml by @babenek in https://github.com/Samsung/CredSweeper/pull/165
    • Version up 1.4.1 by @csh519 in https://github.com/Samsung/CredSweeper/pull/167

    Full Changelog: https://github.com/Samsung/CredSweeper/compare/v1.4.0...v1.4.1

    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Jul 15, 2022)

    What's Changed

    • --ml_validation removed and ML is used by default.
    • All ML packages are installed by default.
    • ZIP archives might be scanned with option --depth (recursive)
    • API validation bugfixes for wrong server response case
    Source code(tar.gz)
    Source code(zip)
  • v1.3.1(Jun 15, 2022)

    What's Changed

    • MlValidator object is created once
    • CredSweeper is installed with pip as CLI tool additionally
    • CredSweeper may be used as github action
    • Added size limit option to skip large files in scan
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(May 18, 2022)

    Changes:

    • change TF to ONNX for faster runtime https://github.com/Samsung/CredSweeper/pull/96
    • change default number of threads to 1 https://github.com/Samsung/CredSweeper/pull/83
    • optional find files by extension https://github.com/Samsung/CredSweeper/pull/101
    • modified JWT pattern to improve metrics https://github.com/Samsung/CredSweeper/pull/104
    • style fixes, type fixes, duplicate code removal
    Source code(tar.gz)
    Source code(zip)
  • v1.2.1(Feb 3, 2022)

    Change list:

    • Limit max tensorflow version by <2.8.0 due to the new tensorflow release that changes API

    Fix issue with tensorflow:

    $ python -m credsweeper --path tests/samples/password --ml_validation
    Traceback (most recent call last):
      File "/home/user/CredSweeper/credsweeper/ml_model/ml_validator.py", line 12, in <module>
        from tensorflow.python.keras.preprocessing.sequence import pad_sequences
    ModuleNotFoundError: No module named 'tensorflow.python.keras.preprocessing'
    

    New tensorflow==2.8.0 might be supported in 1.2.2, 1.3.0 or later versions

    Source code(tar.gz)
    Source code(zip)
    credsweeper-1.2.1-py3-none-any.whl(224.58 KB)
    credsweeper-1.2.1.tar.gz(198.49 KB)
  • v1.2.0(Jan 24, 2022)

    Short description:

    • Make execution time ~5x faster
    • Less CLI output if result is saved to json
    • Fix crash in case of non-utf default system encoding
    • Add option to specify ML threshold (less or more sensitive)
    • New field in the output: float "ml_probability" which is non empty if ml validation was used

    Please note: Faster execution partially based on changes to the credsweeper/rules/config.yaml If you modified this file in your deployment - please apply similar modifications to a newer version!

    Source code(tar.gz)
    Source code(zip)
    credsweeper-1.2.0-py3-none-any.whl(224.84 KB)
  • v1.1.0(Dec 6, 2021)

    Change list:

    • Add diff scan for .patch files. See --diff_path PATH CLI argument
    • Add support for non UTF-8 encoding in files
    • Add 5 new regex rules: Firebase Domain, AWS S3 Bucket, Nonce, Salt, Certificate
    • New severity type: Info
    • Update ML model. Retrain it with new rules (mentioned above) and updated dataset
    • Add batching for ML model, so it would run in parallel (or even on GPU if tensorflow-gpu configured by the user)
    • Add value field for the output

    Add fist release on pypi: https://pypi.org/project/credsweeper/

    Source code(tar.gz)
    Source code(zip)
    credsweeper-1.1.0-py3-none-any.whl(222.25 KB)
  • v1.0.0(Oct 25, 2021)

Owner
Samsung
Samsung Electronics Co.,Ltd.
Samsung
A JupyterLab extension that allows opening files and directories with external desktop applications.

A JupyterLab extension that allows opening files and directories with external desktop applications.

martinRenou 0 Oct 14, 2021
Creates folders into a directory to categorize files in that directory by file extensions and move all things from sub-directories to current directory.

Categorize and Uncategorize Your Folders Table of Content TL;DR just take me to how to install. What are Extension Categorizer and Folder Dumper Insta

Furkan Baytekin 1 Oct 17, 2021
🧹 Create symlinks for .m2ts files and classify them into directories in yyyy-mm format.

?? Create symlinks for .m2ts files and classify them into directories in yyyy-mm format.

Nep 2 Feb 7, 2022
Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

Department for International Trade 206 Jan 2, 2023
Dragon Age: Origins toolset to extract/build .erf files, patch language-specific .dlg files, and view the contents of files in the ERF or GFF format

DAOTools This is a set of tools for Dragon Age: Origins modding. It can patch the text lines of .dlg files, extract and build an .erf file, and view t

null 8 Dec 6, 2022
csv2ir is a script to convert ir .csv files to .ir files for the flipper.

csv2ir csv2ir is a script to convert ir .csv files to .ir files for the flipper. For a repo of .ir files, please see https://github.com/logickworkshop

Alex 38 Dec 31, 2022
This is a junk file creator tool which creates junk files in Internal Storage

This is a junk file creator tool which creates junk files in Internal Storage

KiLL3R_xRO 3 Jun 20, 2021
A tool written in python to generate basic repo files from github

A tool written in python to generate basic repo files from github

Riley 7 Dec 2, 2021
A tool for batch processing large fasta files and accompanying metadata table to upload to repositories via API

Fasta Uploader A tool for batch processing large fasta files and accompanying metadata table to repositories via API The python fasta_uploader.py scri

Centre for Infectious Disease and One Health 1 Dec 9, 2021
Import Python modules from any file system path

pathimp Import Python modules from any file system path. Installation pip3 install pathimp Usage import pathimp

Danijar Hafner 2 Nov 29, 2021
RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem

RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem. These files are exposed either in their original format, or as PDF files that contain your annotations. This lets you manage files in the reMarkable Cloud using the same tools you use on your local system.

Robert Schroll 82 Nov 24, 2022
Maltego transforms to pivot between PE files based on their VirusTotal codeblocks

VirusTotal Codeblocks Maltego Transforms Introduction These Maltego transforms allow you to pivot between different PE files based on codeblocks they

Ariel Jungheit 18 Feb 3, 2022
MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts.

SaveTools MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts. Credits Written by Asteris

null 31 Nov 17, 2022
pydicom - Read, modify and write DICOM files with python code

pydicom is a pure Python package for working with DICOM files. It lets you read, modify and write DICOM data in an easy "pythonic" way.

DICOM in Python 1.5k Jan 4, 2023
shred - A cross-platform library for securely deleting files beyond recovery.

shred Help the project financially: Donate: https://smartlegion.github.io/donate/ Yandex Money: https://yoomoney.ru/to/4100115206129186 PayPal: https:

null 4 Sep 4, 2021
dotsend is a web application which helps you to upload your large files and share file via link

dotsend is a web application which helps you to upload your large files and share file via link

Devocoe 0 Dec 3, 2022
Nintendo Game Boy music assembly files parser into musicxml format

GBMusicParser Nintendo Game Boy music assembly files parser into musicxml format This python code will get an file.asm from the disassembly of a Game

null 1 Dec 11, 2021
useful files for the Freenove Big Hexapod

FreenoveBigHexapod useful files for the Freenove Big Hexapod HexaDogPos is a utility for converting the Freenove xyz co-ordinate system to servo angle

Alex 2 May 28, 2022
A wrapper for DVD file structure and ISO files.

vs-parsedvd DVDs were an error. A wrapper for DVD file structure and ISO files. You can find me in the IEW Discord server

null 7 Nov 17, 2022