Detect secret in source code, scan your repo for leaks. Find secrets with GitGuardian and prevent leaked credentials. GitGuardian is an automated secrets detection & remediation service.

Overview


GitGuardian Shield: protect your secrets with GitGuardian

PyPI Docker Image Version (latest semver) License GitHub stars GitHub Workflow Status CodeFactor Grade Codecov

GitGuardian shield (ggshield) is a CLI application that runs in your local environment or in a CI environment to help you detect more than 300 types of secrets, as well as other potential security vulnerabilities or policy breaks.

GitGuardian shield uses our public API through py-gitguardian to scan and detect potential secrets on files and other text content.

Only metadata such as call time, request size and scan mode is stored from scans using GitGuardian shield, therefore secrets and policy breaks incidents will not be displayed on your dashboard and your files and secrets won't be stored.

You'll need an API Key from GitGuardian to use ggshield.

Add the API Key to your environment variables:

GITGUARDIAN_API_KEY=<GitGuardian API Key>

Currently supported integrations

Table of Contents

  1. Introduction

  2. Installation

  3. Updating

  4. Configuration

    1. Environment Variables
    2. On-premises
    3. Ignoring files
    4. Ignoring a secret
    5. Ignoring a detector
  5. Commands

  6. Pre-commit

    • The pre-commit framework
    • The global and local pre-commit hook
  7. Pre-push

  8. Pre-receive

  9. GitLab

  10. GitHub Actions

  11. Circle CI

  12. Travis CI

  13. Jenkins

  14. Drone

  15. Azure Pipelines

  16. Output

  17. Contributing

  18. License

Installation

Install and update using pip:

$ pip install ggshield

ggshield supports Python 3.6 and newer.

The package should run on MacOS, Linux and Windows.

You'll need an API Key from the GitGuardian dashboard to use ggshield.

Add the API Key to your environment variables:

GITGUARDIAN_API_KEY=<GitGuardian API Key>

Updating

To update ggshield you can add the option -U/--upgrade to the pip install command.

$ pip install -U ggshield

Commands

Usage: ggshield [OPTIONS] COMMAND [ARGS]...

Options:
  -c, --config-path FILE  Set a custom config file. Ignores local and global
                          config files.

  -v, --verbose           Verbose display mode.
  -h, --help              Show this message and exit.

Commands:
  install  Command to install a pre-commit hook (local or global).
  scan     Command to scan various contents.
  ignore   Command to permanently ignore some secrets.

Scan command

ggshield scan is the main command for ggshield, it has a few config options that can be used to override output behaviour.

Usage: ggshield scan [OPTIONS] COMMAND [ARGS]...

  Command to scan various contents.

Options:
  --show-secrets  Show secrets in plaintext instead of hiding them.
  --exit-zero     Always return a 0 (non-error) status code, even if incidents
                  are found.The env var GITGUARDIAN_EXIT_ZERO can also be used
                  to set this option.

  --json             JSON output results  [default: False]
  --all-policies  Present fails of all policies (Filenames, FileExtensions,
                  Secret Detection). By default, only Secret Detection is
                  shown.
  --ignore-default-excludes    Ignore excluded patterns by default. [default:
                               False]

  -v, --verbose   Verbose display mode.
  -o, --output PATH  Route ggshield output to file.
  -h, --help      Show this message and exit.

Commands:
  ci            scan in a CI environment.
  commit-range  scan a defined COMMIT_RANGE in git.
  path          scan files and directories.
  pre-commit    scan as a pre-commit git hook.
  repo          scan a REPOSITORY's commits at a given URL or path.

ggshield scan has different subcommands for each type of scan:

  • CI: scan each commit since the last build in your CI.

    ggshield scan ci

    No options or arguments

  • Commit Range: scan each commit in the given commit range

    Usage: ggshield scan commit-range [OPTIONS] COMMIT_RANGE
    
      scan a defined COMMIT_RANGE in git.
    
      git rev-list COMMIT_RANGE to list several commits to scan. example:
      ggshield scan commit-range HEAD~1...
    
  • Path: scan files or directories with the recursive option.

    Usage: ggshield scan path [OPTIONS] PATHS...
    
      scan files and directories.
    
    Options:
      -r, --recursive  Scan directory recursively
      -y, --yes        Confirm recursive scan
      -h, --help       Show this message and exit.
    
  • Pre-commit: scan every changes that have been staged in a git repository.

    ggshield scan pre-commit

    No options or arguments

  • Repo: scan all commits in a git repository.

    Usage: ggshield scan repo [OPTIONS] REPOSITORY
    
      scan a REPOSITORY at a given URL or path
    
      REPOSITORY is the clone URI or the path of the repository to scan.
      Examples:
    
      ggshield scan repo [email protected]:GitGuardian/ggshield.git
    
      ggshield scan repo /repositories/ggshield
    
  • Docker: scan a Docker image after exporting its filesystem and manifest with the docker save command.

    Usage: ggshield scan docker [OPTIONS] IMAGE_NAME
    
      ggshield will try to pull the image if it's not available locally
    Options:
      -h, --help  Show this message and exit.
    

Install command

The install command allows you to use ggshield as a pre-commit or pre-push hook on your machine, either locally or globally for all repositories.

You will find further details in the pre-commit/pre-push part of this documentation.

Usage: ggshield install [OPTIONS]

  Command to install a pre-commit or pre-push hook (local or global).

Options:
  -m, --mode [local|global]       Hook installation mode  [required]
  -t, --hook-type [pre-commit|pre-push]
                                  Type of hook to install
  -f, --force                     Force override
  -a, --append                    Append to existing script
  -h, --help                      Show this message and exit.

Ignore command

The ignore command allows you to ignore some secrets. For the time being, it only handles the --last-found option that ignore all secrets found by the last run scan command. Under the hood, these secrets are added to the matches-ignore section of your local config file (if no local config file is found, a .gitguardian.yaml file is created).

Warning: Using this command will discard any comment present in the config file.

Usage: ggshield ignore

  Command to ignore all secrets found by the previous scan.

Options:
  -h, --help                 Show this message and exit.
  --last-found               Ignore all secrets found by last run scan

Quota command

Show remaining quota of the workspace.

Usage: ggshield quota [OPTIONS]

  Command to show quotas overview.

Options:
  --json      JSON output results  [default: False]
  -h, --help  Show this message and exit.

Example:

❯ ggshield quota
Quota available: 9440
Quota used in the last 30 days: 560
Total Quota of the workspace: 10000

API Status command

Show API status and version.

Usage: ggshield api-status [OPTIONS]

  Command to show api status.

Options:
  --json      JSON output results  [default: False]
  -h, --help  Show this message and exit.

Example:

❯ ggshield api-status
status: healthy
app-version: 1.27.0-rc.1
secrets-engine-version-version: 2.44.0

Configuration

Configuration in ggshield follows a global>local>CLI configuration scheme.

Meaning options on local overwrite or extend global and options on CLI overwrite or extend local.

ggshield will search for a global config file in the user's home directory (example: ~/.gitguardian.yml on Linux and %USERPROFILE%\.gitguardian on Windows).

ggshield will recognize as well a local config file in the user's working directory (example: ./.gitguardian.yml).

You can also use the option --config-path on the main command to set another config file. In this case, neither local nor global config files will be evaluated (example: ggshield --config-path=~/Desktop/only_config.yaml scan path -r .)

A sample config file can be found at .gitguardian.example

# Exclude files and paths by globbing
paths-ignore:
  - '**/README.md'
  - 'doc/*'
  - 'LICENSE'

# Ignore security incidents with the SHA256 of the occurrence obtained at output or the secret itself
matches-ignore:
  - name:
    match: 530e5a4a7ea00814db8845dd0cae5efaa4b974a3ce1c76d0384ba715248a5dc1
  - name: credentials
    match: MY_TEST_CREDENTIAL

show-secrets: false # default: false

# Set to true if the desired exit code for the CLI is always 0,
# otherwise the exit code will be 1 if incidents are found.
# the environment variable GITGUARDIAN_EXIT_ZERO=true can also be used toggle this behaviour.
exit-zero: false # default: false

# By default only secrets are detected. Use all-policies to toggle this behaviour.
all-policies: false # default: false

api-url: https://api.gitguardian.com # GITGUARDIAN_API_URL environment variable will override this setting

verbose: false # default: false

Notes

Old configuration of matches-ignore with list of secrets is deprecated but still supported :

# Ignore security incidents with the SHA256 of the occurrence obtained at output or the secret itself
matches-ignore:
  - 530e5a4a7ea00814db8845dd0cae5efaa4b974a3ce1c76d0384ba715248a5dc1
  - MY_TEST_CREDENTIAL

Environment Variables

Some configurations on ggshield can be done through environment variables.

Environment variables will override settings set on your config file but will be overridden by command line options.

At startup, ggshield will attempt to load environment variables from different environment files in the following order:

  • path pointed to by the environment variable GITGUARDIAN_DOTENV_PATH
  • .env at your current work directory.
  • .env at the root of the current git directory

Only one file will be loaded of the three.

Reference of current Environment Variables that affect ggshield:

GITGUARDIAN_API_KEY: [Required] API Key for the GitGuardian API.

GITGUARDIAN_API_URL: Custom URL for the scanning API.

GITGUARDIAN_DONT_LOAD_ENV: If set to any value environment variables won't be loaded from a file.

GITGUARDIAN_DOTENV_PATH: If set to a path, `ggshield` will attempt to load the environment from the specified file.

GITGUARDIAN_TIMEOUT: If set to a float, `ggshield scan pre-receive` will timeout
after the specified value. Set to 0 to disable the timeout

GITGUARDIAN_MAX_COMMITS_FOR_HOOK: if set to an int, `ggshield scan pre-receive` and `ggshield scan pre-push`
will not scan more than the specified value of commits in a single scan.

On-premises configuration

GitGuardian shield can be configured to run on your on-premises dashboard, request an API key from your dashboard administrator.

You can modify your environment variables to include:

GITGUARDIAN_API_KEY=<GitGuardian API Key>
GITGUARDIAN_API_URL=<GitGuardian on-premises API URL>

Alternatively to setting the GITGUARDIAN_API_URL environment variable, set the api-url in your .gitguardian.yaml.

Ignoring files

By default ggshield ignores certain files and directories. This list can be found in ggshield/utils.py under IGNORED_DEFAULT_PATTERNS.

You can turn this feature with the flag --ignore-default-excludes or the key ignore-default-excludes in your .gitguardian.yaml

#.gitguardian.yml
# Use default excluded vendors folders
ignore-default-excludes: false # default: false
ggshield scan --ignore-default-excludes path example_file.md

You can also add custom patterns to ignore by using the --exclude option or the key paths-ignore in your .gitguardian.yaml

# .gitguardian.yml
# Exclude files and paths by globbing
paths-ignore:
  - '**/README.md'
  - 'doc/*'
  - 'LICENSE'
ggshield scan --exclude dir/subdir/** path -r dir

Ignoring a secret

Useful for ignoring a revoked test credential or a false positive, there are three ways to ignore a secret with ggshield:

In code

this will also ignore the secret in the GitGuardian dashboard.

Secrets can be ignored in code by suffixing the line with a ggignore comment.

Examples:

def send_to_notifier() -> int:
  return send_slack_message(token="xoxb-23s2js9912ksk120wsjp") # ggignore
func main() {
  high_entropy_test := "A@@E*JN#DK@OE@K(JEN@I@#)" // ggignore
}

Through configuration

Your secret will still show up on the GitGuardian dashboard as potential incident.

You can use the ignore command to ignore the last found secrets in your scan or directly add the ignore SHA that accompanies the incident or one of the secret matches to the configuration file

A secret ignored on the GitGuardian dashboard will still show as a potential incident on ggshield.

Ignoring a detector

Your secret will still show up on the GitGuardian dashboard as potential incident.

You can ignore a detector using the CLI option -b or --banlist-detector or through the configuration:

Examples:

# .gitguardian.yaml
banlisted-detectors: # default: []
  - Generic Password
  - Generic High Entropy Secret
ggshield scan -b "Generic High Entropy Secret" path example_file.md

Pre-commit

The pre-commit framework

In order to use ggshield with the pre-commit framework, you need to do the following steps.

Make sure you have pre-commit installed:

$ pip install pre-commit

Create a .pre-commit-config.yaml file in your root repository:

repos:
  - repo: https://github.com/gitguardian/ggshield
    rev: main
    hooks:
      - id: ggshield
        language_version: python3
        stages: [commit]

Then install the hook with the command:

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

Now you're good to go!

If you want to skip the pre-commit check, you can add -n parameter:

$ git commit -m "commit message" -n

Another way is to add SKIP=hook_id before the command:

$ SKIP=ggshield git commit -m "commit message"

The global and local pre-commit hook

To install pre-commit globally (for all current and future repos), you just need to execute the following command:

$ ggshield install --mode global

It will do the following:

  • check if a global hook folder is defined in the global git configuration
  • create the ~/.git/hooks folder (if needed)
  • create a pre-commit file which will be executed before every commit
  • give executable access to this file

You can also install the hook locally on desired repositories. You just need to go in the repository and execute:

$ ggshield install --mode local

If a pre-commit executable file already exists, it will not be overridden.

You can force override with the --force option:

$ ggshield install --mode local --force

If you already have a pre-commit executable file and you want to use ggshield, all you need to do is to add this line in the file:

$ ggshield scan pre-commit

If you want to try pre-commit scanning through the docker image:

$ docker run -e GITGUARDIAN_API_KEY -v $(pwd):/data --rm gitguardian/ggshield ggshield scan pre-commit

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable of your project or development environment.

Pre-push

Pre-push hooks will not scan more than 50 commits to avoid developer interruption by default.

In case there are more than a 50 commits in a push the hook will be skipped. The amount of commits to scan before skipping the hook can be configured by the key max-commits-for-hook in a GitGuardian configuration file (for example: .gitguardian.yaml).

Pre-push hooks are executed just before git push sends data to the remote host. It will pickup and scan the range of commits between the local ref and the origin ref.

If incidents are detected in this range the push will be cancelled.

With the pre-commit framework

In order to use ggshield with the pre-commit framework, you need to do the following steps.

Make sure you have pre-commit installed:

$ pip install pre-commit

Create a .pre-commit-config.yaml file in your root repository:

repos:
  - repo: https://github.com/gitguardian/ggshield
    rev: main
    hooks:
      - id: ggshield-push
        language_version: python3
        stages: [push]

Then install the hook with the command:

$ pre-commit install --hook-type pre-push
pre-commit installed at .git/hooks/pre-push

With the install command

To install the pre-push hook globally (for all current and future repos), you just need to execute the following command:

$ ggshield install --mode global -t pre-push

It will do the following:

  • check if a global hook folder is defined in the global git configuration
  • create the ~/.git/hooks folder (if needed)
  • create a pre-push file which will be executed before every commit
  • give executable access to this file

You can also install the hook locally on desired repositories. You just need to go in the repository and execute:

$ ggshield install --mode local -t "pre-push"

If a pre-commit executable file already exists, it will not be overridden.

You can force override with the --force option:

$ ggshield install --mode local --force  -t "pre-push"

Or you can append to the existing pre-push script with the --append option:

$ ggshield install --mode local --force  -t "pre-push"

Now you're good to go!

Pre-receive

A pre-receive hook allows you to reject commits from being pushed to a git repository if they do not validate every check. Refer to our learning center for more information.

You can find ggshield's pre-receive hook samples in the doc/pre-receive.sample and doc/pre-receive-docker.sample.

ggshield's pre-receive hook can be skipped if the developer passes the option breakglass to the git push.

For this setting to work the remote must have push options enabled. (git config receive.advertisePushOptions true).

Examples:

$ git push -o breakglass
$ git push --push-option=breakglass

Install ggshield git pre-receive hook

pre-receive.sample

  1. This pre-receive hook requires the host machine to have python>=3.8 and pip installed

  2. Install ggshield from pip: pip install ggshield

  3. Move pre-receive.sample to .git/hooks/pre-receive or to your provider's git hook directory

  4. Do not forget to chmod +x .git/hooks/pre-receive

  5. either set an environment variable machine wide GITGUARDIAN_API_KEY or set it in the .git/hooks/pre-receive as instructed in the sample file.

How do I add ignored matches and use a custom config in this pre-receive hook?

  • Create a gitguardian.yaml somewhere in the system. An example config file is available here
  • Replace in the pre-receive hook
    ggshield scan pre-receive
    with:
    ggshield -c <INSERT path to gitguardian.yaml> scan pre-receive

Install ggshield git pre-receive hook with docker

For the pre-receive hook to work, the directory where the repositories are stored must also be mounted on the container.

pre-receive-docker.sample

  1. This pre-receive hook requires the host machine to have docker installed.
  2. Move pre-receive-docker.sample to .git/hooks/pre-receive
  3. Do not forget to chmod +x .git/hooks/pre-receive
  4. either set an environment variable machine wide GITGUARDIAN_API_KEY or set it in the .git/hooks/pre-receive as instructed in the sample file.

Docker

The GitGuardian Shield docker scanning tool (ggshield scan docker) is used to scan local docker images for secrets present in the image's creation process (dockerfile and build arguments) and in the image's layers' filesystem.

If the image is not available locally on the user's machine, GitGuardian shield will attempt to pull the image using docker pull .

GitLab

You may be interested in using GitGuardian's GitLab integration to ensure full coverage of your GitLab projects as well as full git history scans and reporting.

Configuring GitLab pipelines to use ggshield is as simple as adding a step to your project's pipeline:

stages:
  - scanning

🦉 gitguardian scan:
  image: gitguardian/ggshield:latest
  stage: scanning
  script: ggshield scan ci
  variables:
    GIT_STRATEGY: clone
    GIT_DEPTH: 0

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable in your project settings.

For ggshield to scan every commit in a merge request pipeline the CI must clone the full repository instead of just fetching the branch. The following snippet ensures this behavior.

variables:
GIT_STRATEGY: clone
GIT_DEPTH: 0

GitHub

You may be interested in using GitGuardian's GitHub integration to ensure full coverage of your GitHub projects as well as full git history scans and reporting.

ggshield's support of GitHub comes in the form of GitHub actions.

The action for this repository is hosted at ggshield-action.

Configuring a GitHub workflow to use ggshield is as simple as adding a step to your project's workflow:

name: GitGuardian scan

on: [push, pull_request]

jobs:
  scanning:
    name: GitGuardian scan
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0 # fetch all history so multiple commits can be scanned
      - name: GitGuardian scan
        uses: GitGuardian/ggshield-action@master
        env:
          GITHUB_PUSH_BEFORE_SHA: ${{ github.event.before }}
          GITHUB_PUSH_BASE_SHA: ${{ github.event.base }}
          GITHUB_PULL_BASE_SHA: ${{ github.event.pull_request.base.sha }}
          GITHUB_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
          GITGUARDIAN_API_KEY: ${{ secrets.GITGUARDIAN_API_KEY }}

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY secret in your project settings.

Bitbucket

Bitbucket pipelines do not support commit ranges therefore only your latest commit in a pushed group or in a new branch will be scanned.

Configuring a Bitbucket pipeline to use ggshield is as simple as adding a step to your project's workflow:

pipelines:
  default:
    - step:
        image: gitguardian/ggshield:latest
        services:
          - docker
        script:
          - ggshield scan ci

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable in your project settings.

Circle CI

Circle CI is supported in ggshield through ggshield-orb.

To add ggshield to your pipelines configure your .circleci/config.yml to add the ggshield orb:

orbs:
  ggshield: gitguardian/ggshield

workflows:
  main:
    jobs:
      - ggshield/scan:
          name: ggshield-scan # best practice is to name each orb job
          base_revision: << pipeline.git.base_revision >>
          revision: <
   
    >
   

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable in your project settings.

Travis CI

To add ggshield to your pipelines configure your .travis.yml to add a ggshield scanning job:

jobs:
  include:
    - name: GitGuardian Scan
      language: python
      python: 3.8
      install:
        - pip install ggshield
      script:
        - ggshield scan ci

Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable in your project settings.

Jenkins

To add ggshield to your pipelines configure your Jenkinsfile to add a ggshield stage:

pipeline {
    agent none
    stages {
        stage('GitGuardian Scan') {
            agent {
                docker { image 'gitguardian/ggshield:latest' }
            }
            environment {
                GITGUARDIAN_API_KEY = credentials('gitguardian-api-key')
            }
            steps {
                sh 'ggshield scan ci'
            }
        }
    }
}

Do not forget to add your GitGuardian API Key to the gitguardian-api-key credential in your project settings.

Drone

To add ggshield to your pipelines configure your .drone.yml to add a ggshield stage:

kind: pipeline
type: docker
name: default

steps:
- name: ggshield
  image: gitguardian/ggshield:latest
  commands:
  - ggshield scan ci

Drone CI integration handles only pull-request or merge-request events, push events are not handled. Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY environment variable for your Drone CI workers.

Azure Pipelines

Azure Pipelines does not support commit ranges outside of GitHub Pull Requests, therefore on push events in a regular branch only your latest commit will be scanned. This limitation doesn't apply to GitHub Pull Requests where all the commits in the pull request will be scanned.

To add ggshield to your pipelines configure your azure-pipelines.yml to add a ggshield scanning job:

jobs:
  - job: GitGuardianShield
    pool:
      vmImage: 'ubuntu-latest'
    container: gitguardian/ggshield:latest
    steps:
      - script: ggshield scan ci
        env:
          GITGUARDIAN_API_KEY: $(gitguardianApiKey)

Do not forget to add your GitGuardian API Key to the gitguardianApiKey secret variable in your pipeline settings.

Output

If no secrets or policy breaks have been found, the exit code will be 0:

$ ggshield scan pre-commit

If a secret or other issue is found in your staged code or in your CI, you will have an alert giving you the type of policy break, the filename where the policy break has been found and a patch giving you the position of the policy break in the file:

"XXXXXXXXXXXXXXXXXXXX", |_____AWS Keys_____| 15 | :secret_access_key => "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" |_______________AWS Keys_______________| 16 | } 17 | } ">
$ ggshield scan pre-commit

🛡️  ⚔️  🛡️  2 policy breaks have been found in file production.rb

11 | config.paperclip_defaults = {
12 |     :s3_credentials => {
13 |     :bucket => "XXX",
14 |     :access_key_id => "XXXXXXXXXXXXXXXXXXXX",
                            |_____AWS Keys_____|

15 |     :secret_access_key => "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
                                |_______________AWS Keys_______________|

16 |     }
17 | }

Lines that are too long are truncated to match the size of the terminal, unless the verbose mode is used (-v or --verbose).

Related open source projects

License

GitGuardian shield is MIT licensed.

Comments
  • Turn IgnoredMatch into a dataclass

    Turn IgnoredMatch into a dataclass

    IgnoredMatch is currently defined as a Dict[str, str] with two entries: name and match.

    It should be turned into a dataclass to improve type-safety and make code more expressive. The Cache code must be updated as well, since it also uses the IgnoredMatch dict.

    good first issue tech debt 
    opened by agateau-gg 8
  • Crash when scanning text files containing a `0` byte in it

    Crash when scanning text files containing a `0` byte in it

    GitGuardian Shield Version

    • [x] I can reproduce this bug in the latest version

    Command executed

    ggshield scan path ThirdPartyNotices.rtf

    Describe the bug

    ggshield crashes when scanning the attached file (attached as zip because GitHub does not support rtf files). This is because the final byte of the file is a 0, as can be seen on this hex dump:

    $ tail -n 4 ThirdPartyNotices.rtf | xxd
    00000000: 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d  ================
    00000010: 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d 3d3d  ================
    00000020: 3d3d 3d3d 3d3d 3d3d 3d5c 7061 720d 0a45  =========\par..E
    00000030: 4e44 204f 4620 5c63 6170 7320 2e4e 4554  ND OF \caps .NET
    00000040: 2043 6f6d 7069 6c65 7220 506c 6174 666f   Compiler Platfo
    00000050: 726d 5c63 6170 7330 2020 4e4f 5449 4345  rm\caps0  NOTICE
    00000060: 5320 414e 4420 494e 464f 524d 4154 494f  S AND INFORMATIO
    00000070: 4e5c 7061 720d 0a7d 0d0a 00              N\par..}...
    

    Expected behavior

    ggshield should either:

    • scan the file without failing
    • provide a clear error message about the problem

    Traceback (if available)

    Traceback (most recent call last):
      File "/home/agateau/src/ggshield/ggshield/dev_scan.py", line 168, in path_cmd
        results = files.scan(
      File "/home/agateau/src/ggshield/ggshield/scan/scannable.py", line 139, in scan
        scan = future.result()
      File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
        return self.__get_result()
      File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
        raise self._exception
      File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/home/agateau/src/py-gitguardian/pygitguardian/client.py", line 279, in multi_content_scan
        request_obj = Document.SCHEMA.load(documents, many=True)
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/marshmallow/schema.py", line 719, in load
        return self._do_load(
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/marshmallow/schema.py", line 904, in _do_load
        raise exc
    marshmallow.exceptions.ValidationError: {0: {'document': ['document has null characters']}}
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/agateau/src/ggshield/.venv/bin/ggshield", line 33, in <module>
        sys.exit(load_entry_point('ggshield', 'console_scripts', 'ggshield')())
      File "/home/agateau/src/ggshield/ggshield/cmd.py", line 229, in cli_wrapper
        return_code: int = cli.main(standalone_mode=False)
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/core.py", line 1053, in main
        rv = self.invoke(ctx)
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/core.py", line 754, in invoke
        return __callback(*args, **kwargs)
      File "/home/agateau/src/ggshield/.venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
        return f(get_current_context(), *args, **kwargs)
      File "/home/agateau/src/ggshield/ggshield/dev_scan.py", line 181, in path_cmd
        return handle_exception(error, config.verbose)
      File "/home/agateau/src/ggshield/ggshield/utils.py", line 275, in handle_exception
        raise click.ClickException(str(e))
    click.exceptions.ClickException: {0: {'document': ['document has null characters']}}
    
    bug status:: confirmed 
    opened by agateau-gg 8
  • `DOCKER_COMMAND_TIMEOUT` is not long enough for saving large windows images

    `DOCKER_COMMAND_TIMEOUT` is not long enough for saving large windows images

    GitGuardian Shield Version 1.10.7

    • [x] I can reproduce this bug in the latest version

    Command executed

    ggshield scan docker

    Describe the bug

    There is a 6 minute hardcoded timeout present for all Docker commands: https://github.com/GitGuardian/ggshield/blob/94a1fa0f6402cd1df2dd3dbc5b932862e85f99e5/ggshield/docker.py#L16-L17

    We are finding that some of our Windows containers that we are scanning with the docker scanner are timing out at the image save command.

    Expected behavior

    No error occurs

    Traceback

    Traceback (most recent call last):
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\docker.py", line 53, in docker_save_to_tmp
        subprocess.run(
      File "C:\Python310\lib\subprocess.py", line 503, in run
        stdout, stderr = process.communicate(input, timeout=timeout)
      File "C:\Python310\lib\subprocess.py", line 1149, in communicate
        stdout, stderr = self._communicate(input, endtime, timeout)
      File "C:\Python310\lib\subprocess.py", line 1529, in _communicate
        raise TimeoutExpired(self.args, orig_timeout)
    subprocess.TimeoutExpired: Command '['docker', 'save', 'artifactory.our.domain.name/our-docker/our-dotnet-framework-build:1.0', '-o', 'C:\\Users\\agent\\AppData\\Local\\Temp\\tmp4a2ffxyuggshield\\artifactory.our.domain.name--our-docker--our-dotnet-framework-build:1.0.tar']' timed out after 360 seconds
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
      File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "C:\Python310\lib\runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\Scripts\ggshield.exe\__main__.py", line 7, in <module>
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\cmd.py", line 229, in cli_wrapper
        return_code: int = cli.main(standalone_mode=False)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\core.py", line 1053, in main
        rv = self.invoke(ctx)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\core.py", line 1395, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\core.py", line 754, in invoke
        return __callback(*args, **kwargs)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\click\decorators.py", line 26, in new_func
        return f(get_current_context(), *args, **kwargs)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\docker.py", line 132, in docker_name_cmd
        return handle_exception(error, config.verbose)
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\utils.py", line 271, in handle_exception
        raise e
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\docker.py", line 117, in docker_name_cmd
        archive = str(docker_save_to_tmp(name, temporary_dir))
      File "C:\agent\builds\build-windows-i-00e5eb68f19754db7-1\ourorg\docker-image-builder\ggshield\lib\site-packages\ggshield\docker.py", line 67, in docker_save_to_tmp
        raise click.ClickException('Command "{}" timed out'.format(" ".join(command)))
    click.exceptions.ClickException: Command "docker save artifactory.our.domain.name/our-docker/our-dotnet-framework-build:1.0 -o C:\Users\agent\AppData\Local\Temp\tmp4a2ffxyuggshield\artifactory.our.domain.name--our-docker--our-dotnet-framework-build:1.0.tar" timed out
    
    bug status:: confirmed 
    opened by testworksau 7
  • Make prepush smarter

    Make prepush smarter

    Description

    This PR fixes #303: when ggshield is used a pre-push hook and the user pushes a new branch, ggshield does not know the branch start commit and scans up to 50 commits.

    What has been done

    Some preliminary work has been done first:

    • Adding a way to create real repositories in test code, so that tests are not out of touch with reality.
    • Fixing some tests which were out of touch with reality.

    Then the command was fixed, by taking inspiration from how pre-commit solves this problem, and extending our fix for scanning parent-less commits (#313) to apply there too.

    Finally, the generated hook had to be fixed, because the ggshield secret scan pre-push now uses the command-line arguments, which were not forwarded by the hook we generated.

    Issue

    Fixes #303.

    opened by agateau-gg 5
  • scan repo: do not ignore failures when run from a git checkout

    scan repo: do not ignore failures when run from a git checkout

    If the current working directory is an existing git checkout, then running secret scan repo SOMETHING does not produce an error if SOMETHING is neither the URL of a git repository or a valid path to a git checkout.

    $ ggshield secret scan repo https://example.com/doesnotexist.git
    Error: /tmp/tmpaiyn0u5r is not a git repository
    
    $ mkdir foo
    
    $ cd foo
    
    $ git init
    
    $ ggshield secret scan repo https://example.com/doesnotexist.git
    Scanning Commits  [####################################]  100%
    
    No secrets have been found
    

    This is because we cache the return value of check_git_dir(wd) but this function can be called with wd = None in which case it uses os.getcwd() as the path. If os.getcwd() changes between two check_git_dir() calls then the returned value for the second call is wrong.

    To fix this we:

    • make is_git_dir() do the bulk of the work, remove its default value and cache its results
    • remove the caching decorator from check_git_dir()

    (Doing it this way was less intrusive than removing the default value of check_git_dir())

    opened by agateau-gg 5
  • Do not abort repo scan if a commit fails to scan

    Do not abort repo scan if a commit fails to scan

    The problem

    When ggshield secret scan repo fails to scan a commit, it stops. This is annoying because it forces users to restart the full scan.

    Proposed solution

    The command should note all failures and carry on. At the end it should summarize all failures and exit with an error.

    To be decided: in the case of JSON output, should the failure summary be part of the JSON document or should it be sent directly to stderr?

    feature request 
    opened by agateau-gg 5
  • fix(dev_dependencies): remove types-click from pre-comits

    fix(dev_dependencies): remove types-click from pre-comits

    Remove types-click from pre-commit.

    I suspect the package is no longer necessary (and cause bugs): There is no click folder in https://github.com/python/typeshed/tree/master/stubs (so the link in https://pypi.org/project/types-click/#history is dead).

    I have tried adding types-click as a dev-dependency, but it somehow requires ipython ??? (see https://github.com/GitGuardian/ggshield/runs/6452203176?check_suite_focus=true)

    opened by gg-mmill 5
  • false positive / confusion following email

    false positive / confusion following email

    I'm not sure where to write about this situation, but decided I'll try here.

    I got an email referencing this repo describing that a secret had been exposed in a repo I manage. The referenced exposed secret was actually encrypted by mozilla/sops as intended, so I assume a false positive triggered the email.

    Curious about this project I read up a bit and considered debugging this, I read that an API key was required, but decided against trying to get one since it led to a request to "Act on your behalf". I'm generally concerned about why that was requested.

    I hope this experience is relevant for you to be aware about.

    Email|API key required -|- image|image

    question 
    opened by consideRatio 5
  • Allow --verbose, --debug and --allow-self-signed everywhere

    Allow --verbose, --debug and --allow-self-signed everywhere

    Description

    Click does not allow defining options across groups, this means ggshield -v secret scan path foo.py works but not ggshield secret scan path foo.py -v.

    What has been done

    This PR introduces a new module in cmd: common_options. This module provides an add_common_options() decorator to define Click options we want to make available anywhere on the command-line. Supported options are:

    • -v, --verbose
    • --debug
    • --allow-self-signed

    With these changes, all these commands now work and do the same thing:

    ggshield -v secret scan path foo.py
    ggshield secret -v scan path foo.py
    ggshield secret scan -v path foo.py
    ggshield secret scan path -v foo.py
    ggshield secret scan path foo.py -v
    

    In the next ~episode~ PR

    I have another branch stacked on top of this one to do something similar for secret scan options like --json, --exit-zero, --show-secrets...

    Issue

    Part of #197

    opened by agateau-gg 4
  • Fix issue #336 : Rewriting tests without altered cassette

    Fix issue #336 : Rewriting tests without altered cassette

    I created two return mocks that are called and returns errors using the error that was previously in the casette output. I've decided to put the mocks in tests/conftest.py, not sure that is the best place, let me know if something is more suitable.

    opened by Kariss83 4
  • `ggshield auth login` error: `RuntimeError: dictionary changed size during iteration`

    `ggshield auth login` error: `RuntimeError: dictionary changed size during iteration`

    GitGuardian Shield Version

    • [x] I can reproduce this bug in the latest version

    Command executed

    ggshield auth login
    

    Describe the bug I'm a new user and wish to install ggshield CLI on my mac.

    On my mac (macOS monterey 12.5), I installed ggshield via brew install gitguardian/tap/ggshield

    This installed version 1.13.0.

    ggshield --version
    ggshield, version 1.13.0
    

    I then ran ggshield auth login and received the following error:

    Traceback (most recent call last):
      File "/opt/homebrew/bin/ggshield", line 8, in <module>
        sys.exit(main())
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/cmd/main.py", line 156, in main
        return cli.main(args, prog_name="ggshield", standalone_mode=not show_crash_log)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/click/core.py", line 1053, in main
        rv = self.invoke(ctx)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/click/core.py", line 1656, in invoke
        super().invoke(ctx)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/click/core.py", line 754, in invoke
        return __callback(*args, **kwargs)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
        return f(get_current_context(), *args, **kwargs)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/cmd/main.py", line 117, in cli
        config = Config(config_path)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/core/config/config.py", line 40, in __init__
        self.user_config, self._config_path = UserConfig.load(config_path=config_path)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/core/config/user_config.py", line 144, in load
        user_config._update_from_file(local_config_path)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/core/config/user_config.py", line 167, in _update_from_file
        obj = UserV1Config.load_v1(data)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/core/config/user_config.py", line 215, in load_v1
        UserV1Config.matches_ignore_to_dict(data)
      File "/opt/homebrew/Cellar/ggshield/1.13.0/libexec/lib/python3.9/site-packages/ggshield/core/config/user_config.py", line 262, in matches_ignore_to_dict
        for idx, match in enumerate(matches_ignore):
    RuntimeError: dictionary changed size during iteration
    

    This is consistent behaviour, running ggshield auth login multiple times results in the same error message.

    Expected behavior

    I was expecting to login and not be presented with any errors.

    Screenshots

    Traceback (if available)

    bug status:: triage 
    opened by iainsproat 4
  • git guardian github action bug

    git guardian github action bug

    Environment

    • ggshield version: latest
    • Operating system (Linux, macOS, Windows): windows
    • Python version: 3.9

    Describe the bug

    Getting this error when I run git guardian in github action workflow

    Permission denied: '/github/home/.cache'

    Steps to reproduce:

    1. copy this github action code at https://dashboard.gitguardian.com/workspace/335609/settings/workspace/integrations/github_actions into github action workflow yml file
    2. Commit/push to dev branch
    3. observe github action

    Actual result:

    Permission denied: '/github/home/.cache'

    Expected result:

    result showing scan and pass/fail for any vulnerability

    If applicable, add logs or screenshots to help explain your problem.

    bug question status:: triage 
    opened by karkir0003 3
  • The CloudSmith entitlement token isn't detected as the secret

    The CloudSmith entitlement token isn't detected as the secret

    Hello The cloudsmith entitlement token isn't detected.

    Honestly speaking it's a bit tricky to detect. The token is relatively short without clear identifier , for example DqhyrhnfAOky.

    Any chance to add it anyway do the detection engine ?

    bug status:: triage 
    opened by alexku7 2
  • Enhance reporting with metrics

    Enhance reporting with metrics

    Is your feature request related to a problem? Please describe.

    Even is ggshield is properly deployed on developer machines etc... we don't get much insights and metrics about "how much is caught", which could tell us a bit more about bad habits, even though they don't leak.

    Describe the solution you'd like

    GGshield is authenticated to the API, why not ping back the type of secret and author back ?

    Describe alternatives you've considered

    There is a PR ready to merge in DefectDojo, which makes me think I could modify the pre-commit hook, and report to defectdojo if something is found, instead of using a Gitguardian API.

    Additional context

    I have discussed that with you in your offices :)

    feature request 
    opened by Gby56 2
  • pre-receive timeout is not precise enough

    pre-receive timeout is not precise enough

    Environment

    • ggshield version: 1.13.6
    • Operating system (Linux, macOS, Windows): Linux
    • Operating system version: Ubuntu 20.04
    • Python version: 3.8

    Describe the bug

    ggshield secret scan pre-receive is supposed to stop after a predefined timeout, which defaults to 4.5s by default. However it can sometimes take more than twice this time to stop.

    Steps to reproduce:

    1. Setup a local bare repository with a pre-receive hook calling ggshield
    2. Clone it
    3. Make your internet connection slow enough to hit the timeout (For example use wondershaper to limit the bandwidth of your network interface to 2KBps)
    4. Measure the timeout

    Actual result:

    It took 10s for the hook to stop on my machine.

    Expected result:

    The difference between the defined timeout and the actual delay should be less than 1s.

    Possible explanation

    I suspect this is because the timeout is implemented using a thread, and Python GIL gets in the way. If this is the case, then that bug can be solved by running the scan in a different process.

    bug status:: confirmed 
    opened by agateau-gg 0
  • Content is not correctly censored

    Content is not correctly censored

    Environment

    • ggshield version: latest
    • Operating system (Linux, macOS, Windows): Linux
    • Operating system version:
    • Python version: 3.10

    Describe the bug

    Steps to reproduce:

    1. create file secret.txt with the following content (note the starting empty line)
    
    password = 123nlsdkjfsaodi09ufsdf
    
    # known
    secret = "sk_live_epISFDSkdeXmn5asTvb7RHAi"
    
    # new
    secret = "sk_live_epISFDSkdeXmn5bnKvb7RHBn"
    
    1. Run command ggshield secret scan path secret.txt

    Actual result:

    Screenshot from 2022-11-07 14-17-34

    Expected result:

    Secret censored correctly

    Note: when in commit (pre-commit, pre-push etc) the content is censored correctly. Same bug with ggshield secret scan archive ...

    bug good first issue status:: confirmed 
    opened by alina-tuholukova-gg 0
Releases(v1.14.2)
  • v1.14.2(Dec 15, 2022)

    Changed

    • It is now possible to use generic command-line options like --verbose anywhere on the command line and scan options anywhere after the scan word (#197).

    • ggshield iac scan now shows the severity of the detected vulnerabilities.

    Fixed

    • If a file containing secrets has been committed in two different branches, then ggshield secret scan repo would show 4 secrets instead of 2. This has been fixed (#428).

    • ggshield now uses different error codes when a scan succeeds but finds problems and when a scan does not finish (#404).

    • ggshield now correctly handles the case where git is not installed (#329).

    Source code(tar.gz)
    Source code(zip)
    ggshield-1.14.2-1.x86_64.rpm(3.00 MB)
    ggshield-1.14.2.pyz(3.04 MB)
    ggshield_1.14.2-1_amd64.deb(3.00 MB)
  • v1.14.1(Nov 16, 2022)

  • v1.14.0(Nov 16, 2022)

    Added

    • ggshield scan commands now accept the --ignore-known-secrets option. This option is useful when working on an existing code-base while secrets are being remediated.

    • ggshield learned a new secret scan command: docset. This command can scan any content as long as it has been converted into our new docset file format.

    Changed

    • ggshield auth login --method=token can now read its token from the standard input.

    Fixed

    • ggshield now prints clearer error messages if the .gitguardian.yaml file is invalid (#377).

    • When used with the pre-commit framework, ggshield would sometimes scan commits with many files more than once. This has been fixed.

    Source code(tar.gz)
    Source code(zip)
    ggshield-1.14.0-1.x86_64.rpm(3.12 MB)
    ggshield-1.14.0.pyz(3.16 MB)
    ggshield_1.14.0-1_amd64.deb(3.12 MB)
  • v1.13.6(Oct 19, 2022)

  • v1.13.4(Oct 12, 2022)

    Added

    • ggshield now checks for update once a day and notifies the user if a new version is available. This check can be disabled with the --no-check-for-updates command-line option (#299).

    Changed

    • Scanning Git repositories is now faster.

    • ggshield secret scan path now shows a progress bar.

    • When used as a pre-push or pre-receive hook, ggshield no longer scans more commits than necessary when a new branch is pushed (#303, #369).

    Fixed

    • ggshield no longer declares two separate instances if the instance URL is set with and without a trailing slash (#357).

    • Fixed a regression where ggshield would not load the .env from the current working directory.

    • ggshield no longer silently ignores network issues.

    Source code(tar.gz)
    Source code(zip)
    ggshield-1.13.4-1.x86_64.rpm(3.11 MB)
    ggshield-1.13.4.pyz(3.15 MB)
    ggshield_1.13.4-1_amd64.deb(3.11 MB)
  • v1.13.3(Sep 14, 2022)

    🔧 Improvements

    • Scanning is faster, thanks to ggshield making better use of available CPU cores (#356).

    • ggshield secret scan commands now use less memory on large file trees and skip more binary files (#353).

    🐛 Bug fixes

    • ggshield no longer fails when scanning a repository with only one commit in it (#313).

    • ggshield now properly decodes and detects secrets in texts stored in UTF-16, UTF-32 and others, with or without BOM (#355).

    • ggshield no longer ignores errors when running ggshield secret scan repo on a non-existing repository from within an existing repository (#351).

    Source code(tar.gz)
    Source code(zip)
    ggshield-1.13.3-1.x86_64.rpm(1.79 MB)
    ggshield-1.13.3.pyz(1.80 MB)
    ggshield_1.13.3-1_amd64.deb(1.78 MB)
  • v1.13.2(Aug 29, 2022)

  • v1.13.1(Aug 29, 2022)

    🐛 Bug fixes

    • secret scan repo no longer aborts if ggshield fails to parse a git commit (#323).
    • iac commands now return a non-zero exit code in case of errors.
    • Merge commits are now correctly parsed (#322).
    • ggshield no longer creates its cache directory when running secret scan pre-receive (#306).
    • The iac scan command now has an help message.
    • The command "mode" for secret scan ci is now correctly reported in server statistics (#320).

    🔧 Chores

    • The minimum version of pygitguardian is now 1.3.5 (#319).
    Source code(tar.gz)
    Source code(zip)
    ggshield-1.13.1-1.x86_64.rpm(1.78 MB)
    ggshield-1.13.1.pyz(1.80 MB)
    ggshield_1.13.1-1_amd64.deb(1.78 MB)
  • v1.13.0(Jul 20, 2022)

    🚀 New features

    • The new ggshield iac scan command lets you detect vulnerabilities in your Infrastructure as Code files. Note that this feature is experimental for now.

    🔧 Improvements

    • ggshield secret scan repo now continues scanning if a commit fails to scan (#267).
    • ggshield now provides a --debug option to help diagnose problems.

    🐛 Bug fixes

    • ggshield now uses stderr to report all progress and warnings, making it possible to pipe commands generating JSON such as secret scan --json to tools like jq.
    • The file-system banlist of ggshield secret scan docker has been improved (#193).
    • ggshield secret scan pre-receive and ggshield secret scan repo no longer fail on filenames containing spaces (#273, #296).
    • ggshield secret scan repo no longer floods the terminal with "No secrets found" messages (#265).
    • The commands used by the pre-commit hooks and by the GitHub action no longer use the deprecated ggshield scan syntax.

    📜 Deprecations

    • The configuration file format has changed, learn more about this change and how to adapt to it from the documentation.
    Source code(tar.gz)
    Source code(zip)
    ggshield-1.13.0-1.x86_64.rpm(1.78 MB)
    ggshield-1.13.0.pyz(1.79 MB)
    ggshield_1.13.0-1_amd64.deb(1.77 MB)
  • v1.12.0(May 30, 2022)

    🚀 Features

    • It is now easier to setup ggshield, thanks to the new ggshield auth commands
    • ggshield deb and rpm packages are now available on Cloudsmith

    🐛 Bug fixes

    • Do not stop when scanning a file ending with \0 (#155)
    • Send verbose messages to stderr to allow piping JSON output (#157)
    • Print a message when no secrets are detected (#117)

    📜 Deprecation

    • The ggshield scan commands are deprecated, use ggshield secret scan instead
    • The ggshield ignore command is deprecated, use ggshield secret ignore instead
    Source code(tar.gz)
    Source code(zip)
    ggshield-1.12.0-1.x86_64.rpm(1.70 MB)
    ggshield-1.12.0.pyz(1.71 MB)
    ggshield_1.12.0-1_amd64.deb(1.70 MB)
  • v1.11.0(Mar 29, 2022)

    :rocket: Features

    • 30f93f1 (cmd) #113 scan archive
    • 2b471b0 (cmd) #113 scan pypi packages

    :bug: Bug fixes

    • cfbaf17 (docker) fix crash when scanning a non-local image
    • 4224efd (import) standardize import format
    • 9e782e1 (path) remove an useless IO operation
    • b53da7c (setup) provide useful error message when API key is not latin-1 (#101)
    • e844671 (tests) Make test_text_output pass on Windows
    • ba91b89 (tests) fix test_status_cli on Windows
    • 8b87f64 (tests) skip failing Windows tests for now
    • 736febb (tests) fix some tests failing because test files were not created
    • c9e63fd (tests) fix test_scan.py::test_cd_context_manager on Windows
    • 518a946 (tests) make Docker tests pass on Windows
    • bc5780a only show backtrace if GITGUARDIAN_CRASH_LOG is set
    • 4a67841 fix Click warning about deprecated symbol
    • 0082660 fix pre-receive hook error handling with GitLab Web UI

    :wrench: Chores and Improvements

    • b09b5bd (ci) fix build job always running on Ubuntu
    • d9231d2 (doc) fix outdated comments
    • 76f4cd4 (gitlab-ci) remove unused definition
    • d09c9a7 (release) 1.11.0
    • 2574ff6 (tests) remove empty test file
    • 8907772 (tests) add test for is_filepath_excluded()
    • e59da7c (typing) fully type Config fields
    • f5fcbc9 (typing) Remove typing errors in tests
    • 20bdfe2 remove unnecessary backslashes
    • d92e632 pin dependency versions
    • 679d451 remove unused show_secrets arg from get_lines_from_content()
    • f03580c improve typing
    • 6ae73f7 move validity translation to text_utils
    • 80dd202 add brew command for easy install on :apple: macos (#171)
    Source code(tar.gz)
    Source code(zip)
    ggshield.pyz(1.48 MB)
  • v1.10.8(Jan 31, 2022)

    :rocket: Features

    • b865bff (docker) add a --docker-timeout option (#161)

    :bug: Bug fixes

    • 0e0f64e (cache) fix uncaught Exception when cache is created on a read-only FS
    • 26830aa (docker) fix "docker save" failing on Windows (#161)
    • 307d662 (docker) fix crash on Windows when saving Docker images (#160)
    • e8cdd94 (tests) remove platform specific code
    • fee7b07 (tests) support macos tmp dir to make tests pass
    • e7047f4 (ci) stop to tag every Docker images as latest

    :wrench: Chores and Improvements

    • 387c8a4 (release) 1.10.8
    • 22690b7 (docs) add GITGUARDIAN_API_URL to pre-receive hook

    :package: Other

    • 316d72f Update doc/pre-receive.sample

    Co-authored-by: Aurelien Gateau [email protected]

    • cedee18 Merge pull request #162 from GitGuardian/jeremy/-/ci-docker-images-tag

    fix(ci): stop to tag every Docker images as latest

    • 94a1fa0 Refactor output code (#156)

    chore(output_handler): refactor output handling

    • Remove unnecessary empty methods from OutputHandler
    • Move code writing to stdout or to the specified file to OutputHandler
    • Move code computing the exit code to OutputHandler
    • Rename concrete classes from ${FORMAT}Handler to ${FORMAT}OutputHandler and their files from ${format}_output.py to ${format}_output_handler.py
    • Add docstrings to OutputHandler methods
    • 621a6bb (pre-commit) add description to hooks (#153)
    • 8796fbc Unbreak unit tests (#154)
    • chore: update cassettes and snapshots

    Fixes #148

    Use GitGuardian test tokens instead of SendGrid Key.

    Do not reuse cassettes between tests, it's confusing and feels brittle.

    Do not activate pytest-socket by default, only turn it on in the CI. This way one can update VCR cassettes by removing the cassette files, no need to edit the pyproject.toml file. Proposed changes still need to not do any network access for the CI to pass.

    Make exclude tests easier to understand: all runs are now expected to exit with 0 and we check stdout to verify if the normally ignored files are there or not. Factorize code creating the normally ignored test file.

    • 4ac156b (README) fix azure pipelines link (#152)
    • 23b6402 fix(ci) no secret found no more displayed if secrets found (#147)
    • fix(ci) do not display "no secret found" when secrets are found

    Co-authored-by: [email protected] [email protected] Co-authored-by: agateau-gg [email protected]

    • d36f9f9 Merge pull request #146 from GitGuardian/phililippe/fix-tests-tmp-macos

    fix(tests): fix tests failure on macOS because of /tmp dir

    Source code(tar.gz)
    Source code(zip)
    ggshield.pyz(1.34 MB)
  • v1.10.7(Nov 25, 2021)

  • v1.10.6(Nov 25, 2021)

    :rocket: Features

    • f7c4718 (cache) only create/update cache if there are new entries
    • 2bd8013 (release) add .pyz support through shiv. implements #109

    :bug: Bug fixes

    • 0c6857e (release) fix release notary not finding the latest release

    :wrench: Chores and Improvements

    • f71d5b1 (release) 1.10.6
    • 27e0b3c (deps) update hooks and dev deps
    • d8ee302 (workflow) update jobs to push ggshield formula on both gitguardian's tap

    :package: Other

    • 0bfdcbe Merge pull request #145 from pierrelalanne/plalanne/-/handle-new-homebrew-tap

    Update jobs to push ggshield formula on both GitGuardian's taps

    :wrench: Chores and Improvements

    • 4b62880 (release) 1.10.7
    Source code(tar.gz)
    Source code(zip)
    ggshield.pyz(1.36 MB)
  • v1.10.5(Nov 18, 2021)

    :bug: Bug fixes

    • c4ac7a3 (patch_parsing) lines starting with backslash
    • acb3ce6 (filter) stop to list FS to exclude files to scan

    :wrench: Chores and Improvements

    • 2097e18 (release) 1.10.5

    :package: Other

    • b2e1a1b Merge pull request #139 from GitGuardian/jeremy/-/exclude-patterns-with-regex

    fix(filter): stop to list FS to exclude files to scan

    • ba71c6f Merge pull request #137 from GitGuardian/mmillet/-/fix_lines_parsing

    fix(patch_parsing): lines starting with backslash

    Source code(tar.gz)
    Source code(zip)
  • v1.10.4(Nov 5, 2021)

    :bug: Bug fixes

    • 69204a9 (pre-receive) never run standalone mode. fixes exit 1 on interrupt

    :wrench: Chores and Improvements

    • db6b367 (release) 1.10.4
    Source code(tar.gz)
    Source code(zip)
  • v1.10.3(Nov 2, 2021)

    :rocket: Features

    • 53754e0 (pre-receive) try to diff against HEAD
    • 146d982 (pre-receive) add max-hook configurable by env variable

    :wrench: Chores and Improvements

    • a9759ed (version) bump patch version
    Source code(tar.gz)
    Source code(zip)
  • v1.10.2(Nov 2, 2021)

    :rocket: Features

    • 2795c3c (validity) add validity display for json output
    • 04798c1 (validity) add validity support for text_output
    • 5bffadd (json_output) add pre and post line indices
    • 434d26d (pre-receive) add pre-receive timeout configuration env variable and make sure the exit code is 0

    :bug: Bug fixes

    • 5024f35 (api-status) add API URL to api-status
    • 31849a8 (git) ignore invalid utf8

    :wrench: Chores and Improvements

    • 8d3a84c (version) bump patch version
    • d2000a4 (actions) ensure brew is updated

    :package: Other

    • de45d6b (README) marketing fix
    • 4bac36d (ggshield) document GITGUARDIAN_CRASH_LOG
    • d4f1605 (README) small documentation fixes
    Source code(tar.gz)
    Source code(zip)
  • v1.10.1(Oct 29, 2021)

  • v1.10.0(Oct 29, 2021)

    :rocket: Features

    • 8990fda (scan) add --ignore-default-excludes to exclude some patterns by default
    • 0fe22e7 (scan) add a --exclude option

    :bug: Bug fixes

    • 089bae3 (ci) fix commit range for gitlab merge request pipelines
    • 0df0e2f (scan) path should not ignore not added files even when running on a git repository
    • 9ffb488 (config) do not turn keys of type set into list in update_config()

    :wrench: Chores and Improvements

    • 4af8e69 (readme) add information on ignoring files and detectors
    • 787aa88 (deps) update pygitguardian
    • 69f4fea (version) bump minor version
    • de838d8 (tests) add ipdb as the default debugger for tests
    • 5101734 (test) add TEST_GITGUARDIAN_API_KEY to CONTRIBUTING.md

    :package: Other

    • 428ea05 (gitlab) add warning about MR pipelines
    • 765c04a (ggshield) add validity check and reorganize tests
    • d83f208 (ggshield) add pytest-socket
    • f3c480e Merge pull request #118 from Sayrus/fix-documentation-samples

    docs(pre-receive): fix filenames and python version

    • 88fa8bb (contributing) add instruction for message check pre-commit hook
    • 00c8dbf (scan) make the help of repo cmd clearer
    • 2bcccf2 (README) update pre-receive docs
    • 1fb057b (pre-receive) fix filenames and python version
    Source code(tar.gz)
    Source code(zip)
  • v1.9.0(Oct 11, 2021)

    :rocket: Features

    • 1220413 (pre-receive) add gitlab web detection
    • 2ce2bfa (pre-receive) add pre-receive timeout
    • f6e39c3 (pre-receive) return 0 if breakglassed
    • 87a82c9 (pre-receive) add specific pre-receive-mode

    :bug: Bug fixes

    • de3daaf (pre-receive) git status -C doesnt work on bare repos
    • 9c26571 (ci) disable cache for scan ci command

    :wrench: Chores and Improvements

    • fb9d8ff (ci) create release
    • a09a3c1 (ggshield) bump hardcoded version
    • 291af0b (cache) do not create cache file until we need it
    • cd927ab (pre-receive) add explanation to magic number.
    • 9f842ba (ggshield) refactor scan exception handle

    :package: Other

    • 86c47dd (pre-receive) improve menu items
    • 4838736 (pre-receive) add pre-receive docs
    • bcd55ae (pre-receive) add pre-receive tests
    • 263393e (prepush) move prepush tests to own file
    Source code(tar.gz)
    Source code(zip)
  • v1.8.2(Sep 23, 2021)

    :bug: Bug fixes

    • 90b3b59 (ggshield) fix typo in api-status menu

    :wrench: Chores and Improvements

    • 2189af3 (ggshield) drop python 3.6 support
    • 4ba1b38 (pipfile) update dev dependencies
    • e52c430 (pre-commit) pre-commit run --all files
    • ac3ae4a (deps) update dependencies

    :package: Other

    • 255c42d (ggshield) add new options to config and readme
    Source code(tar.gz)
    Source code(zip)
  • v1.8.1(Aug 20, 2021)

    :wrench: Chores and Improvements

    • 8563248 (setup) fix missing yaspin dependency
    • 6e27e63 (setup) fix setup spacing
    • a15d0e6 (pipfile) fix lock
    Source code(tar.gz)
    Source code(zip)
  • v1.8.0(Aug 20, 2021)

    :rocket: Features

    • Add option to banlist detectors
    • Add config to banlist detectors
    • Add spinner during docker save

    :bug: Bug fixes

    • Fixed progress bar during scan processing

    :wrench: Chores and Improvements

    Source code(tar.gz)
    Source code(zip)
  • v1.7.3(Jul 20, 2021)

  • v1.7.2(Jul 15, 2021)

    :bug: Bug fixes

    • 0d4601f (output) add a new lines
    • f0035d5 (cmd) rename deprecated callback
    • 98ae6b8 (docker) handle incomplete docker config

    :wrench: Chores and Improvements

    • 626f289 (release) update patch
    • 61a7247 (ggshield) add 3.9 compatibility tag

    :package: Other

    • 71cc491 Merge pull request #90 from GitGuardian/jeremy/2356/scan-docker-with-incomplete-config

    fix(docker): handle incomplete docker config

    Source code(tar.gz)
    Source code(zip)
  • v1.7.1(Jul 12, 2021)

    :rocket: Features

    • 32602ee (cmd) add an option to ignore ssl

    :bug: Bug fixes

    • 40d0299 (urlib) do not warn if the user knows he's disabling SSL verification

    :wrench: Chores and Improvements

    • 0448765 (ggshield) rename actions
    • 1f63dec (ggshield) rename most references in repo
    • f7bf1fc (actions) fix typo
    • f5ce573 (actions) provide a valid token
    Source code(tar.gz)
    Source code(zip)
  • v1.7.0(Jul 6, 2021)

    :rocket: Features

    • fd7bde7 (chore) update py-gitguardian
    • c6b392c (output) add secrets engine version to scan results
    • 419d331 (status_cmd) add the api-status command

    :bug: Bug fixes

    • 573faf8 (json) fix line display on JSON output

    :wrench: Chores and Improvements

    • 4ef8054 (README) add new commands to readme and bump version
    • 1fe6884 (README) add update instructions
    • 078e60f (docker) update dockerfile to 3.9
    • 5cc6ec0 (github) add brew update

    :package: Other

    • 4852859 (policy-breaks) test user display mode
    Source code(tar.gz)
    Source code(zip)
  • v1.6.1(Jun 23, 2021)

    :bug: Bug fixes

    • b0416a5 (prepush) fix stdin input single line and new tree scanning

    :wrench: Chores and Improvements

    • 0345c8f (release) update patch

    :package: Other

    • 32184d5 (cmd) improve test correctness
    Source code(tar.gz)
    Source code(zip)
Owner
GitGuardian
GitGuardian is a developers-first solution scanning GitHub activity in real-time for API secret tokens, database credentials, certificates, ...
GitGuardian
pyGinit is a command line tools that help you to initialize your current project a local git repo and remote repo

pyGinit pyGinit is a command line tools that help you to initialize your current project a local git repo and remote repo Requirements Requirements be

AlphaBeta 15 Feb 26, 2022
A CLI Application to detect plagiarism in Source Code Files.

Plag Description A CLI Application to detect plagiarism in Source Code Files. Features Compare source code files for plagiarism. Extract code features

default=dev 2 Nov 10, 2022
A python program to detect your emotion and suggest a playlist.

mood_music A python program to detect your emotion and suggest a playlist. This program is written in python using opencv,FER() and tensorflow This pr

Aditya_Sai 1 Jan 2, 2022
👻 Ghoul is an easy to use information service, allowing you to get/add information on someone or something directly from your terminal.

?? Ghoul is an easy to use information service, allowing you to get/add information on someone or something directly from your terminal. It c

Billy 11 Nov 10, 2021
Find your broken links, so users don't.

PyAnchor Dead links are an annoyance for websites with an extensive amount of content. A side from the negative impact on SEO, dead links are an annoy

Ricky White 61 Dec 14, 2022
A simple CLI application helps you to find giant files that are eating up your system storage

Large file finder Sometimes it's very hard to find if some giant files are eating up your system storage. We might need to hunt those down. This simpl

Rahul Baruri 5 Nov 18, 2022
Fylm is a wonderful automated command line app for organizing your film media.

Overview Fylm is a wonderful automated command line app for organizing your film media. You can pronounce it Film or File 'em, whichever you like! It

Brandon Shelley 30 Dec 5, 2022
Phishing-Detective is a command line application for Windows 10 built to detect a phishing site from two url's

Phishing-Detective Phishing-Detective is a command line application for Windows 10 built to detect a phishing site from two url's How it works A simpl

null 2 Jun 23, 2022
AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

Introduction AthenaCLI is a command line interface (CLI) for the Athena service that can do auto-completion and syntax highlighting, and is a proud me

dbcli 192 Jan 7, 2023
A command line tool to query source code from your current Python env

wxc wxc (pronounced "which") allows you to inspect source code in your Python environment from the command line. It is based on the inspect module fro

Clément Robert 13 Nov 8, 2022
A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool

Privateer A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool How

Shreyash Chavan 2 Apr 4, 2022
triggercmd is a CLI client for the TRIGGERcmd cloud service.

TriggerCMD CLI client triggercmd is a CLI client for the TRIGGERcmd cloud service. installation the triggercmd package is available in PyPI. to instal

Gustavo Soares 7 Oct 18, 2022
swarmexec executes command in swarm service

Swarmexec swarmexec executes command in swarm service Install pip install git+https://github.com/filimon43g/swarmexec.git Config In swarm_config.ini

Phil 2 Nov 23, 2021
spid-sp-test is a SAML2 SPID/CIE Service Provider validation tool that can be executed from the command line.

spid-sp-test spid-sp-test is a SAML2 SPID/CIE Service Provider validation tool that can be executed from the command line. This tool was born by separ

Developers Italia 30 Nov 8, 2022
ServX | Bash Command as a Service

ServX | Bash Command as a Service Screenshots Instructions for running Run python3 servx.py. COMPATIBILITY TESTED ON ARCHLINUX(x64) & DEBIAN(x64) ONLY

ARPSyndicate 2 Mar 11, 2022
GitFun - A Simplified Automated CLI tool for GIT😃

GitFun A Simplified Automated CLI tool for GIT, It's for Lazy Developers and Newbies ?? Table of contents GitFun Installation Usage Development Contri

Astaqc 8 Feb 22, 2022
Chopper: An Automated Security Headers Analyzer

____ _ _ / ___| |__ ___ _ __ _ __ ___ _ __| | | | | '_ \ / _ \| '_ \| '_ \ / _ \ '__| | | |___| | | | (_) |

Kamran Saifullah (Frog Man) 2 Nov 27, 2022
open a remote repo locally quickly

A command line tool to peek a remote repo hosted on github or gitlab locally and view it in your favorite editor. The tool handles cleanup of the repo once you exit your editor.

Rahul Nair 44 Dec 16, 2022
A simple automation script that logs into your kra account and files your taxes with one command

EASY_TAX A simple automation script that logs into your kra account and files your taxes with one command Currently works for Chrome users. Will creat

leon koech 13 Sep 23, 2021