Audits Python environments and dependency trees for known vulnerabilities

Overview

pip-audit

CI

pip-audit is a prototype tool for scanning Python environments for packages with known vulnerabilities. It uses the Python Packaging Advisory Database (https://github.com/pypa/advisory-db) as a source of vulnerability reports.

This project is developed by Trail of Bits with support from Google. This is not an official Google product.

Development steps

git clone https://github.com/trailofbits/pip-audit && cd pip-audit
make dev && source env/bin/activate
pip-audit --help

Release process

Releases of pip-audit are managed via bump and GitHub Actions.

# default release (patch bump)
make release

# override the default
# vX.Y.Z -> vX.Y.Z-rc.0
make release BUMP_ARGS="--pre rc.0"

# vX.Y.Z -> vN.0.0
make release BUMP_ARGS="--major"

make release will fail if there are any untracked changes in the source tree.

If make release succeeds, you'll see an output like this:

RUN ME MANUALLY: git push origin main && git push origin vX.Y.Z

Run that last command sequence to complete the release.

Comments
  • 0.0.4 test failure: fixture 'vuln_service' not found

    0.0.4 test failure: fixture 'vuln_service' not found

    When running the self tests for pip-audit-0.0.4 using py.test, I see:

    ___________________________________________________________ ERROR at setup of test_audit ___________________________________________________________
    file /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py, line 10
      def test_audit(vuln_service, dep_source):                                                                                                         
    E       fixture 'vuln_service' not found                                  
    >       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, 
    record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory                                        
    >       use 'pytest --fixtures [testpath]' for help on them.              
                                                                                                                                                        
    /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py:10                                                                                
    _______________________________________________________ ERROR at setup of test_audit_dry_run _______________________________________________________
    file /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py, line 32                                                                     
      def test_audit_dry_run(monkeypatch, vuln_service, dep_source):                                                                                    
    E       fixture 'vuln_service' not found                                                                                                            
    >       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, 
    record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
    >       use 'pytest --fixtures [testpath]' for help on them.                                                                                        
                                                                                                                                                        
    /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py:32
    
    bug 
    opened by 0-wiz-0 35
  • scan without resolving/downloading dependencies

    scan without resolving/downloading dependencies

    maybe i'm misunderstanding, but it seems that pip-audit cannot scan the contents of a requirements file in the simplest way imaginable: for each entry, check if the specified version has known vulnerabilities.

    currently, running against a requirements file with exclusively exact version matches (foo==x.y), like this

    $ pip-audit -r requirements.txt 
    

    … downloads and (temporarily) installs various packages, which not only takes forever, but also eventually fails because some packages cannot be found – they come from a private index server, which is a different issue.

    as a work-around, installing everything (also downloads and installs) and then using pip-audit --local seems to work, but this is not a convenient solution.

    i understand that in the general case, requirements files may not be exhaustive and may not have exact versions, in which case running a resolver etc makes total sense, but that's not always the case.

    some background: the use case is that my requirements files are produced by pip-tools and hence constitute the exact list of pinned versions used by my application, and i would like to integrate pip-audit into a continuous integration job. i also run pip-compile (from pip-tools) in that job to warn about potential package upgrades, some of which come from private index servers, and it would be great if pip-audit can be added to it.

    enhancement 
    opened by wbolster 19
  • Fails to parse data-requires-python with a `*` char

    Fails to parse data-requires-python with a `*` char

    Bug description

    pip-audit raises an InvalidSpecifier error when trying to parse a data-requires-python key containing a * from pypi. Seems related to this old issue : https://github.com/pypa/pip-audit/issues/138

    Reproduction steps

    I found the nltk package wich contains this link in its pypi page raising the mentioned error:

    <a 
      href="https://files.pythonhosted.org/packages/98/06/de681159e6750d0a215c2126e784504e177896afddf5a68cba42ebe42355/nltk-3.6-py3-none-any.whl#sha256=718de6908f538db19a77f96b9e6f5f586b0892d7de5eea32e71f2a2535ed8657" 
      data-requires-python="&gt;=3.5.*" 
    >
      nltk-3.6-py3-none-any.whl
    </a><br />
    

    To reproduce, we only need to launch a pip-audit with this dependency.

    echo "nltk" > requirements.txt
    pip-audit -v --requirement requirements.txt                       
    

    Screenshots and logs

    DEBUG:pip_audit._cli:parsed arguments: Namespace(cache_dir=None, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, dry_run=False, extra_index_urls=[], fix=False, format=<OutputFormatChoice.Columns: 'columns'>, ignore_vulns=[], index_url='https://pypi.org/simple/', local=False, no_deps=False, output=PosixPath('stdout'), paths=[], progress_spinner=<ProgressSpinnerChoice.On: 'on'>, project_path=None, require_hashes=False, requirements=[<_io.TextIOWrapper name='test_require.txt' mode='r' encoding='UTF-8'>], skip_editable=False, strict=False, timeout=15, verbose=True, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>)
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/nltk/" in the cache
    DEBUG:cachecontrol.controller:Current age based on date: 1186
    DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/nltk/ HTTP/1.1" 304 0
    Traceback (most recent call last):
      File "/usr/local/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_cli.py", line 448, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_audit.py", line 67, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_service/interface.py", line 155, in query_all
        for spec in specs:
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 116, in collect
        for _, dep in self._collect_cached_deps(filename, reqs):
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
        for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/interface.py", line 88, in resolve_all
        yield (req, self.resolve(req))
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
        result = self.resolver.resolve([req])
      File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 372, in resolve
        self._add_to_criteria(self.state.criteria, r, parent=None)
      File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 168, in _add_to_criteria
        candidates=build_iter_view(matches),
      File "/usr/local/lib/python3.8/site-packages/resolvelib/structs.py", line 169, in build_iter_view
        matches = list(matches)
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 362, in find_matches
        [
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 362, in <listcomp>
        [
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 203, in get_project_from_indexes
        yield from get_project_from_index(index_url, session, project, extras, timeout, state)
      File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 249, in get_project_from_index
        spec = SpecifierSet(py_req)
      File "/usr/local/lib/python3.8/site-packages/packaging/specifiers.py", line 700, in __init__
        parsed.add(Specifier(specifier))
      File "/usr/local/lib/python3.8/site-packages/packaging/specifiers.py", line 234, in __init__
        raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
    packaging.specifiers.InvalidSpecifier: Invalid specifier: '>=3.5.*'
    

    Platform information

    • Linux
    • pip-audit 2.4.9
    • Python 3.8.2
    • pip 22.3.1
    bug component:dep-sources 
    opened by j0ack 18
  • Incompatibility with packaging>=22.0: cannot import name 'LegacyVersion' from 'packaging.version'

    Incompatibility with packaging>=22.0: cannot import name 'LegacyVersion' from 'packaging.version'

    Bug description

    pip-audit is incompatible with latest version of packaging, due to removed LegacyVersion

    Reproduction steps

    1. pip install pip-audit
    2. Check that packaging version is >= 22.0
    3. Run pip freeze | pip-audit --no-deps -r - to check the currently installed packages

    Expected behavior

    pip-audit installs a compatible version of packaging

    Screenshots and logs

    Traceback (most recent call last):
      File "/usr/local/bin/pip-audit", line 5, in <module>
        from pip_audit._cli import audit
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 17, in <module>
        from pip_audit._audit import AuditOptions, Auditor
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 10, in <module>
        from pip_audit._dependency_source import DependencySource
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/__init__.py", line 14, in <module>
        from .requirement import RequirementSource
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 19, in <module>
        from pip_requirements_parser import InstallRequirement, InvalidRequirementLine, RequirementsFile
      File "/usr/local/lib/python3.9/site-packages/pip_requirements_parser.py", line 73, in <module>
        from packaging.version import LegacyVersion
    ImportError: cannot import name 'LegacyVersion' from 'packaging.version' (/usr/local/lib/python3.9/site-packages/packaging/version.py)
    

    Platform information

    • OS name and version: docker, python:3.9-alpine
    • pip-audit version (pip-audit -V): 2.4.7
    • Python version (python -V or python3 -V): 3.9.15
    • pip version (pip -V or pip3 -V): 22.0.4
    bug 
    opened by haydngreatnews 15
  • pip-audit failure using --index-url

    pip-audit failure using --index-url

    Bug description

    Hey guys sorry I'm actually not sure if this is a bug or I'm doing something wrong. pip-audit is failing with a custom pypi index-url option

    Reproduction steps

    pip-audit -r requirements.txt --index-url https://custom-repo.com/api/pypi/pypi-all/simple

    Expected behavior

    Expect this to download the packages from the custom pypi repo as expected. for example pip install -r requirements.txt --index-url https://custom-repo.com/api/pypi/pypi-all/simple works as expected

    Screenshots and logs

    $ pip-audit -v \
        --format json \
        --progress-spinner off \
        --requirement ./_pip_audit.txt \
        --index-url https://custom-repo.com/api/pypi/pypi-all/simple \
        --output /report/pip-audit.json
    DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='./_pip_audit.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Json: 'json'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=PosixPath('/cache'), progress_spinner=<ProgressSpinnerChoice.Off: 'off'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://custom-repo.com/api/pypi/pypi-all/simple', extra_index_urls=[], skip_editable=False, no_deps=False, output=<_io.TextIOWrapper name='/report/pip-audit.json' mode='w' encoding='UTF-8'>, ignore_vulns=[])
    DEBUG:cachecontrol.controller:Looking up "https://custom-repo.com/api/pypi/pypi-all/simple/django" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): custom-repo.com:443
    DEBUG:urllib3.connectionpool:https://custom-repo.com:443 "GET /api/pypi/pypi-all/simple/django HTTP/1.1" 302 0
    DEBUG:cachecontrol.controller:Status code 302 not in (200, 203, 300, 301, 308)
    DEBUG:cachecontrol.controller:Looking up "https://custom-repo.com:443/api/pypi/pypi-all/simple/django/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://custom-repo.com:443 "GET /api/pypi/pypi-all/simple/django/ HTTP/1.1" 200 None
    DEBUG:cachecontrol.controller:Updating cache with response from "https://custom-repo.com:443/api/pypi/pypi-all/simple/django/"
    Traceback (most recent call last):
      File "/usr/local/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 432, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 66, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 150, in query_all
        for spec in specs:
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 114, in collect
        for _, dep in self._collect_cached_deps(filename, reqs):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 326, in _collect_cached_deps
        for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
        yield (req, self.resolve(req))
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 75, in resolve
        result = self.resolver.resolve([req])
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 521, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 402, in resolve
        failure_causes = self._attempt_to_pin_criterion(name)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
        criteria = self._get_updated_criteria(candidate)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
        for requirement in self._p.get_dependencies(candidate=candidate):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 384, in get_dependencies
        return candidate.dependencies
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 121, in dependencies
        self._dependencies = list(self._get_dependencies())
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 103, in _get_dependencies
        deps: List[str] = self.metadata.get_all("Requires-Dist", [])
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in metadata
        self._metadata = self._get_metadata_for_wheel()
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 128, in _get_metadata_for_wheel
        data = self._session.get(self.url, timeout=self._timeout).content
      File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
        return self.request("GET", url, **kwargs)
      File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 573, in request
        prep = self.prepare_request(req)
      File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 484, in prepare_request
        p.prepare(
      File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 368, in prepare
        self.prepare_url(url, params)
      File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 439, in prepare_url
        raise MissingSchema(
    requests.exceptions.MissingSchema: Invalid URL '../../packages/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl#sha256=6b1de6886cae14c7c44d188f580f8ba8da05750f544c80ae5ad43375ab293cd5': No scheme supplied. Perhaps you meant http://../../packages/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl#sha256=6b1de6886cae14c7c44d188f580f8ba8da05750f544c80ae5ad43375ab293cd5?
    

    Platform information

    • Linux Centos 7, Docker python:3.9-alpine
    • pip-audit 2.4.5
    • Python 3.9.15
    • pip 22.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

    Additional context

    Running without the index-url doesn't work either

    $ pip-audit -v \
        --format json \
        --progress-spinner off \
        --cache-dir /cache \
        --requirement _pip_audit.txt \
        --output /report/pip-audit.json
    DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='_pip_audit.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Json: 'json'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=PosixPath('/cache'), progress_spinner=<ProgressSpinnerChoice.Off: 'off'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://pypi.org/simple', extra_index_urls=[], skip_editable=False, no_deps=False, output=<_io.TextIOWrapper name='/report/pip-audit.json' mode='w' encoding='UTF-8'>, ignore_vulns=[])
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/django HTTP/1.1" 301 112
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/django"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/django/ HTTP/1.1" 200 48676
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/django/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl HTTP/1.1" 200 8094697
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/asgiref HTTP/1.1" 301 113
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/asgiref"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/asgiref/ HTTP/1.1" 200 10369
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/asgiref/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/sqlparse HTTP/1.1" 301 114
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/sqlparse"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/sqlparse/ HTTP/1.1" 200 4081
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/sqlparse/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl HTTP/1.1" 200 22881
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl HTTP/1.1" 200 42768
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/django/4.1.3/json" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/django/4.1.3/json HTTP/1.1" 200 2287
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/django/4.1.3/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/asgiref/3.5.2/json" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/asgiref/3.5.2/json HTTP/1.1" 200 4383
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/asgiref/3.5.2/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/sqlparse/0.4.3/json" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/sqlparse/0.4.3/json HTTP/1.1" 200 2044
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/sqlparse/0.4.3/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dvm" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dvm HTTP/1.1" 301 109
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dvm"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dvm/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dvm/ HTTP/1.1" 200 463
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dvm/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl HTTP/1.1" 200 18333
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/numpy HTTP/1.1" 301 111
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/numpy"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/numpy/ HTTP/1.1" 200 211789
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/numpy/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pymc3" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pymc3 HTTP/1.1" 301 111
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pymc3"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pymc3/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pymc3/ HTTP/1.1" 200 6667
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pymc3/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pandas HTTP/1.1" 301 112
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pandas"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pandas/ HTTP/1.1" 200 129114
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pandas/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tqdm" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tqdm HTTP/1.1" 301 110
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tqdm"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tqdm/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tqdm/ HTTP/1.1" 200 26943
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tqdm/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow-probability" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow-probability HTTP/1.1" 301 128
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow-probability"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow-probability/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow-probability/ HTTP/1.1" 200 4916
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow-probability/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow HTTP/1.1" 301 116
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow/ HTTP/1.1" 200 97476
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/seaborn" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/seaborn HTTP/1.1" 301 113
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/seaborn"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/seaborn/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/seaborn/ HTTP/1.1" 200 4702
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/seaborn/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl HTTP/1.1" 200 872225
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/arviz" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/arviz HTTP/1.1" 301 111
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/arviz"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/arviz/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/arviz/ HTTP/1.1" 200 5657
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/arviz/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cachetools" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachetools HTTP/1.1" 301 116
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cachetools"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cachetools/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachetools/ HTTP/1.1" 200 6943
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cachetools/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/deprecat" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/deprecat HTTP/1.1" 301 114
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/deprecat"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/deprecat/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/deprecat/ HTTP/1.1" 200 1768
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/deprecat/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dill" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dill HTTP/1.1" 301 110
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dill"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dill/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dill/ HTTP/1.1" 200 3943
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dill/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/fastprogress" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/fastprogress HTTP/1.1" 301 118
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/fastprogress"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/fastprogress/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/fastprogress/ HTTP/1.1" 200 5148
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/fastprogress/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy" in the cache
    DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy/" in the cache
    DEBUG:cachecontrol.controller:Current age based on date: 23
    DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
    DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
    DEBUG:cachecontrol.controller:600 > 23
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas" in the cache
    DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas/" in the cache
    DEBUG:cachecontrol.controller:Current age based on date: 23
    DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
    DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
    DEBUG:cachecontrol.controller:600 > 23
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/patsy" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/patsy HTTP/1.1" 301 111
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/patsy"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/patsy/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/patsy/ HTTP/1.1" 200 2061
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/patsy/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/scipy" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/scipy HTTP/1.1" 301 111
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/scipy"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/scipy/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/scipy/ HTTP/1.1" 200 114256
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/scipy/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/semver" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/semver HTTP/1.1" 301 112
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/semver"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/semver/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/semver/ HTTP/1.1" 200 7095
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/semver/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/theano-pymc" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/theano-pymc HTTP/1.1" 301 117
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/theano-pymc"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/theano-pymc/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/theano-pymc/ HTTP/1.1" 200 1513
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/theano-pymc/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/typing-extensions" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/typing-extensions HTTP/1.1" 301 123
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/typing-extensions"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/typing-extensions/" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/typing-extensions/ HTTP/1.1" 200 4800
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/typing-extensions/"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl HTTP/1.1" 200 12901
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz HTTP/1.1" 200 1810180
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 104, in post_setup
        run(package_install_cmd, state=self._state)
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_subprocess.py", line 51, in run
        raise CalledProcessError(f"{pretty_args} exited with {process.returncode}")
    pip_audit._subprocess.CalledProcessError: python -m pip install /tmp/tmpft40uuu8/Theano-PyMC-1.1.2.tar.gz exited with 1
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/usr/local/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 432, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 66, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 150, in query_all
        for spec in specs:
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 114, in collect
        for _, dep in self._collect_cached_deps(filename, reqs):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 326, in _collect_cached_deps
        for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
        yield (req, self.resolve(req))
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 75, in resolve
        result = self.resolver.resolve([req])
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 521, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 402, in resolve
        failure_causes = self._attempt_to_pin_criterion(name)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
        criteria = self._get_updated_criteria(candidate)
      File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
        for requirement in self._p.get_dependencies(candidate=candidate):
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 384, in get_dependencies
        return candidate.dependencies
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 121, in dependencies
        self._dependencies = list(self._get_dependencies())
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 103, in _get_dependencies
        deps: List[str] = self.metadata.get_all("Requires-Dist", [])
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 96, in metadata
        self._metadata = self._get_metadata_for_sdist()
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 165, in _get_metadata_for_sdist
        ve.create(ve_dir)
      File "/usr/local/lib/python3.9/venv/__init__.py", line 78, in create
        self.post_setup(context)
      File "/usr/local/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 106, in post_setup
        raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
    pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpbniri7fx/bin/python', '-m', 'pip', 'install', '/tmp/tmpft40uuu8/Theano-PyMC-1.1.2.tar.gz']
    
    bug component:dep-sources 
    opened by fi0rini 15
  • osv reports pyyaml needs upgrade when it is already upgraded

    osv reports pyyaml needs upgrade when it is already upgraded

    Hello, I can't tell is this is because osv is corrupt and y'all can't do anything about it or if pip-audit can't do version number comparisons. I'm still learning, but I can't find a way to skip this.

    It only takes one unresolvable false positive and I have to disable a tool like this from the build server. safety has a --ignore switch, does pip-audit have one?

    $ pip-audit -s osv
    Found 1 known vulnerabilities in 1 packages
    Name   Version ID             Fix Versions
    ------ ------- -------------- ------------
    pyyaml 5.4.1   PYSEC-2021-142 5.4
    
    $ pip freeze | grep PyY
    PyYAML==5.4.1
    
    upstream 
    opened by matthewdeanmartin 15
  • json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 13246)

    json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 13246)

    I just ran pip-audit-0.0.4 for the first time and got an error:

    Traceback (most recent call last):
      File "/usr/pkg/lib/python3.9/site-packages/pip_audit/dependency_source/pip.py", line 64, in collect
        for (_, dist) in pip_api.installed_distributions(local=self._local).items():
      File "/usr/pkg/lib/python3.9/site-packages/pip_api/_installed_distributions.py", line 83, in installed_distributions
        return _new_installed_distributions(local)
      File "/usr/pkg/lib/python3.9/site-packages/pip_api/_installed_distributions.py", line 71, in _new_installed_distributions
        for raw_dist in json.loads(result):
      File "/usr/pkg/lib/python3.9/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
      File "/usr/pkg/lib/python3.9/json/decoder.py", line 340, in decode
        raise JSONDecodeError("Extra data", s, end)
    json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 13246)
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/usr/pkg/bin/pip-audit-3.9", line 33, in <module>
        sys.exit(load_entry_point('pip-audit==0.0.4', 'console_scripts', 'pip-audit')())
      File "/usr/pkg/lib/python3.9/site-packages/pip_audit/cli.py", line 196, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/usr/pkg/lib/python3.9/site-packages/pip_audit/audit.py", line 60, in audit
        yield from self._service.query_all(specs)
      File "/usr/pkg/lib/python3.9/site-packages/pip_audit/service/interface.py", line 82, in query_all
        for spec in specs:
      File "/usr/pkg/lib/python3.9/site-packages/pip_audit/dependency_source/pip.py", line 72, in collect
        raise PipSourceError("failed to list installed distributions") from e
    pip_audit.dependency_source.pip.PipSourceError: failed to list installed distributions
    
    bug 
    opened by 0-wiz-0 15
  • Crash on `package @ git+...` dependencies

    Crash on `package @ git+...` dependencies

    Bug description

    Using package @ git+... dependencies crashes pip-audit with a traceback, when I'd expect it to output the usual report with those packages listed by name and skip-reason if unauditable.

    Reproduction steps

    # Some packages exist, and all is well
    shed == 0.10.5
    
    # Others *don't* exist or can't be fetched, that's reported nicely
    this_might_exist_off_pypi == 0.0.1
    
    # But if you have this awful kind of dep, you'll get a traceback!
    hypothesis @ git+https://github.com/HypothesisWorks/hypothesis.git@bb6b55ad8d#subdirectory=hypothesis-python
    
    pip-audit --no-deps -r requirements.txt
    

    Platform information

    • OS name and version: macOS Monterey
    • pip-audit version (pip-audit -V): pip-audit 2.4.4
    • Python version (python -V or python3 -V): Python 3.10.6
    • pip version (pip -V or pip3 -V): pip 22.3
    bug component:dep-sources 
    opened by Zac-HD 14
  • Dependency resolution fails to install `cyclonedx-python-lib-0.11.1`

    Dependency resolution fails to install `cyclonedx-python-lib-0.11.1`

    Bug description

    If I audit a requirements file that transitively depends on cyclonedx-python-lib, I see a pip install failure.

    Reproduction steps

    1. Make a requirements.txt that looks like this (or any dependency that relies on cyclonedx-python-lib):
    pip-audit==1.0.0
    
    1. Run pip-audit -r requirements.txt
    2. Observe the failure

    Expected behavior

    All dependencies are audited properly.

    Screenshots and logs

    (.ve) tetsuo@Alexs-MacBook-Pro test % pip-audit -r requirements.txt
    - Installing package in isolated environment
    Traceback (most recent call last):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 90, in post_setup
        subprocess.run(
      File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmppf9hai1y/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpykqq2afm/cyclonedx-python-lib-0.11.1/LICENSE']' returned non-zero exit status 1.
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/Users/tetsuo/Workspace/test/.ve/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_cli.py", line 240, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
        yield from self._service.query_all(specs)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
        for spec in specs:
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 62, in collect
        for _, deps in self.resolver.resolve_all(iter(req_values)):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
        yield (req, self.resolve(req))
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 41, in resolve
        result = self.resolver.resolve([req])
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 373, in resolve
        failure_causes = self._attempt_to_pin_criterion(name)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
        criteria = self._get_updated_criteria(candidate)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
        for requirement in self._p.get_dependencies(candidate=candidate):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 293, in get_dependencies
        return candidate.dependencies
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 112, in dependencies
        self._dependencies = list(self._get_dependencies())
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in _get_dependencies
        deps: List[str] = self.metadata.get_all("Requires-Dist", [])
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 87, in metadata
        self._metadata = self._get_metadata_for_sdist()
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 170, in _get_metadata_for_sdist
        ve.create(ve_dir)
      File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/venv/__init__.py", line 78, in create
        self.post_setup(context)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 97, in post_setup
        raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
    pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmppf9hai1y/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpykqq2afm/cyclonedx-python-lib-0.11.1/LICENSE']
    (.ve) tetsuo@Alexs-MacBook-Pro test % vim requirements.txt
    (.ve) tetsuo@Alexs-MacBook-Pro test % PIP_AUDIT_LOGLEVEL=debug pip-audit -r requirements.txt
    DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.On: 'on'>, timeout=15)
    - Installing package in isolated environment
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-audit HTTP/1.1" 301 215
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-audit/ HTTP/1.1" 200 2843
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/16/b2/c8a8208030b89825cdc1a4f2edaabf2076ba38ab07e77213fc299cf74d99/pip_audit-1.0.0-py3-none-any.whl HTTP/1.1" 200 49177
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-api HTTP/1.1" 301 213
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-api/ HTTP/1.1" 200 4058
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packaging HTTP/1.1" 301 215
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packaging/ HTTP/1.1" 200 7379
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress HTTP/1.1" 301 214
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress/ HTTP/1.1" 200 1044
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/resolvelib HTTP/1.1" 301 216
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/resolvelib/ HTTP/1.1" 200 3179
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/html5lib HTTP/1.1" 301 214
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/html5lib/ HTTP/1.1" 200 3097
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachecontrol HTTP/1.1" 301 218
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachecontrol/ HTTP/1.1" 200 4358
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/lockfile HTTP/1.1" 301 214
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/lockfile/ HTTP/1.1" 200 990
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cyclonedx-python-lib HTTP/1.1" 301 226
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cyclonedx-python-lib/ HTTP/1.1" 200 4903
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/2a/68/d8412d1e0d70edf9791cbac5426dc859f4649afc22f2abbeb0d947cf70fd/progress-1.6.tar.gz HTTP/1.1" 200 7842
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip HTTP/1.1" 301 109
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 200 18015
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress HTTP/1.1" 301 214
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress/ HTTP/1.1" 200 1044
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools HTTP/1.1" 301 216
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools/ HTTP/1.1" 200 104565
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/e8/28/b6b91e2dd3dbc964db396650703b922d5f0047a32c7f1af5c3c6c662b5cb/pip_api-0.0.23-py3-none-any.whl HTTP/1.1" 200 107323
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip HTTP/1.1" 301 109
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 200 18015
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/6c/dd/a834df6482147d48e225a49515aabc28974ad5a4ca3215c18a882565b028/html5lib-1.1-py2.py3-none-any.whl HTTP/1.1" 200 112173
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/six HTTP/1.1" 301 109
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/six/ HTTP/1.1" 200 4429
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/webencodings HTTP/1.1" 301 218
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/webencodings/ HTTP/1.1" 200 891
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/d3/39/b7cd9ef1be03ac33e71f76837a23d59842b016e5159cf5aff30c0b340907/CacheControl-0.12.10-py2.py3-none-any.whl HTTP/1.1" 200 20297
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requests HTTP/1.1" 301 114
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requests/ HTTP/1.1" 200 18544
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/msgpack HTTP/1.1" 301 213
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/msgpack/ HTTP/1.1" 200 28426
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/c8/22/9460e311f340cb62d26a38c419b1381b8593b0bb6b5d1f056938b086d362/lockfile-0.12.2-py2.py3-none-any.whl HTTP/1.1" 200 13564
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/0c/51/51b9438d2813d8252cade62650347b38fbf692e761cc0511d8c81adfdd31/cyclonedx_python_lib-0.11.1-py3-none-any.whl HTTP/1.1" 200 127708
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packageurl-python HTTP/1.1" 301 223
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packageurl-python/ HTTP/1.1" 200 3928
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requirements-parser HTTP/1.1" 301 225
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requirements-parser/ HTTP/1.1" 200 822
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools HTTP/1.1" 301 216
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools/ HTTP/1.1" 200 104565
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/13/ea/5720270a0e984e5dd238ec8e75a8acccb042ac97bcb215be9e2cb339dc1f/cyclonedx-python-lib-0.11.1.tar.gz HTTP/1.1" 200 98048
    Traceback (most recent call last):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 90, in post_setup
        subprocess.run(
      File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpapwlw08o/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpz7uf8d_w/cyclonedx-python-lib-0.11.1/LICENSE']' returned non-zero exit status 1.
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/Users/tetsuo/Workspace/test/.ve/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_cli.py", line 240, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
        yield from self._service.query_all(specs)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
        for spec in specs:
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 62, in collect
        for _, deps in self.resolver.resolve_all(iter(req_values)):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
        yield (req, self.resolve(req))
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 41, in resolve
        result = self.resolver.resolve([req])
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 373, in resolve
        failure_causes = self._attempt_to_pin_criterion(name)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
        criteria = self._get_updated_criteria(candidate)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
        for requirement in self._p.get_dependencies(candidate=candidate):
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 293, in get_dependencies
        return candidate.dependencies
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 112, in dependencies
        self._dependencies = list(self._get_dependencies())
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in _get_dependencies
        deps: List[str] = self.metadata.get_all("Requires-Dist", [])
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 87, in metadata
        self._metadata = self._get_metadata_for_sdist()
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 170, in _get_metadata_for_sdist
        ve.create(ve_dir)
      File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/venv/__init__.py", line 78, in create
        self.post_setup(context)
      File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 97, in post_setup
        raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
    pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpapwlw08o/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpz7uf8d_w/cyclonedx-python-lib-0.11.1/LICENSE']
    

    Platform information

    • OS name and version: macOS 12.0.1
    • pip-audit version (pip-audit -V): 1.0.0
    • Python version (python -V or python3 -V): 3.9.7
    • pip version (pip -V or pip3 -V): 21.3.1
    bug component:dep-sources 
    opened by tetsuo-cpp 13
  • `pip` sometimes includes debug messages on `stdout`

    `pip` sometimes includes debug messages on `stdout`

    This is an abbreviated copy of #115 and #116, since those have been filled up with additional debugging.

    Summary: In rare cases (currently N=1), we fail to collect environmental dependencies from PipSource. PipSource uses pip-api internally, which boils down to pip list --format=json.

    What seems to happen is that the pip list --format=json command doesn't emit just JSON. Instead, it also emits a trailing log message that looks like this:

    Given no hashes to check 181 links for project 'pip': discarding no candidates
    

    According to pip's source code, that message is produced by a call to log.debug, which should be going to stderr instead of stdout. pip-api only uses stdout from pip subprocesses, so this is a strong indicator that the two streams are being mixed and/or confused somewhere, potentially with a logging override. The original reporter also traced the process and confirmed that the log was produced on stdout.

    The original reporter found this on Python 3.9, pip version 20.3.3. I was unable to reproduce it locally, and they were also unable to reliably reproduce it locally.

    bug 
    opened by woodruffw 13
  • pip-audit not auditing the same package versions as pip installs.

    pip-audit not auditing the same package versions as pip installs.

    Bug description

    My understanding of pip-audit is that, when given a requirements.txt file, it would audit the same versions of the same packages that pip would install. This doesn't seem to be the case.

    I have a requirements.txt file containing just the cryptography module with no version constraint specified.

    If I create a venv and install the package then it installs:

    • cryptography 36.0.2
    • cffi 1.15.0 (as cryptography has a requirement for cffi>=1.1.2)
    • pcparser 2.21

    If I run pip-audit -r requirement.txt then it tries to install cffi version 1.0.2-2 rather than 1.15.0 and fails with:

    pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpffo5omkp/bin/python3', '-m', 'pip', 'install', '/tmp/tmp4dbeewpp/cffi-1.0.2-2.tar.gz']

    If I change the requirements.txt file to contain cryptography==36.0.2, the same thing happens.

    Likewise, if I append cffi==1.15.0 to the requirements.txt file pip-audit still tries, and fails, to use cffi 1.0.2-2

    Reproduction steps

    Running in Ubuntu 22.04 Docker container with just python3, venv, git installed. Running as a non-root user in the container I install pip-audit, either via pip, or directly from the git repo Create requirements.txt containing just cryptography Create a venv, install cryptography package from requirements.txt using pip Run pip list to confirm package versions installed as listed above (specifically cffi 1.15.0) Run pip-audit -r requirements.txt pip-audit tries to install cffi 1.0.2.2 and fails, as above

    Expected behavior

    pip-audit should audit the same packages and the same versions of the packages as pip install installs pip-audit should not fail

    Screenshots and logs

    Dockerfile used:

    # Start with up-to-date Ubuntu
    
    FROM ubuntu:22.04
    
    # User to run with
    
    ARG BUILD_USER=build
    ARG BUILD_UID=1000
    ARG BUILD_GID=1000
    ARG HOME_DIR=/home/build
    
    # Update & upgrade, install minimal Python setup
    
    RUN apt update
    RUN apt upgrade --yes
    RUN apt install --yes apt-utils
    RUN apt install --yes python3 python3-pip python3-venv git
    
    # Add the user
    
    RUN adduser --disabled-password --gecos '' ${BUILD_USER}
    USER ${BUILD_UID}
    
    WORKDIR ${HOME_DIR}
    
    # Install pip-audit
    
    #RUN pip install --upgrade --no-warn-script-location pip-audit
    RUN python3 -m pip install git+https://github.com/trailofbits/pip-audit
    

    Contents of requirements.txt file:

    cryptography
    

    Output from pip list in venv after installing cryptography package:

    Package      Version
    ------------ -------
    cffi         1.15.0
    cryptography 36.0.2
    pip          22.0.2
    pycparser    2.21
    setuptools   59.6.0
    

    Output from pip-audit -v -r requirements.txt:

    DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.On: 'on'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://pypi.org/simple', extra_index_urls=[], skip_editable=False)
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cryptography HTTP/1.1" 301 118
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cryptography"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography/" in the cache
    DEBUG:cachecontrol.controller:Current age based on date: 153
    DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
    DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
    DEBUG:cachecontrol.controller:600 > 153
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl HTTP/1.1" 200 4716191
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cffi HTTP/1.1" 301 110
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cffi"
    DEBUG:cachecontrol.controller:Caching permanent redirect
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi/" in the cache
    DEBUG:cachecontrol.controller:Current age based on date: 153
    DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
    DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
    DEBUG:cachecontrol.controller:600 > 153
    DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz HTTP/1.1" 200 317417
    DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
    DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    Traceback (most recent call last):
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 103, in post_setup
        run(package_install_cmd, state=self._state)
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_subprocess.py", line 51, in run
        raise CalledProcessError(f"{pretty_args} exited with {process.returncode}")
    pip_audit._subprocess.CalledProcessError: python3 -m pip install /tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz exited with 1
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/home/build/.local/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_cli.py", line 357, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_audit.py", line 66, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 142, in query_all
        for spec in specs:
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 98, in collect
        for _, deps in self._resolver.resolve_all(iter(req_values)):
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
        yield (req, self.resolve(req))
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 73, in resolve
        result = self.resolver.resolve([req])
      File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 481, in resolve
        state = resolution.resolve(requirements, max_rounds=max_rounds)
      File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 373, in resolve
        failure_causes = self._attempt_to_pin_criterion(name)
      File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
        criteria = self._get_updated_criteria(candidate)
      File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
        for requirement in self._p.get_dependencies(candidate=candidate):
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 341, in get_dependencies
        return candidate.dependencies
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 119, in dependencies
        self._dependencies = list(self._get_dependencies())
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 101, in _get_dependencies
        deps: List[str] = self.metadata.get_all("Requires-Dist", [])
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in metadata
        self._metadata = self._get_metadata_for_sdist()
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 162, in _get_metadata_for_sdist
        ve.create(ve_dir)
      File "/usr/lib/python3.10/venv/__init__.py", line 78, in create
        self.post_setup(context)
      File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 105, in post_setup
        raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
    pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpjk63y53b/bin/python3', '-m', 'pip', 'install', '/tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz']
    

    Platform information

    • OS name and version: Ubuntu 22.04 (in Docker container)
    • pip-audit version (pip-audit -V): pip-audit 2.1.0
    • Python version (python -V or python3 -V): Python 3.10.3
    • pip version (pip -V or pip3 -V): pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
    bug component:dep-sources upstream 
    opened by skilleter 12
  • Environmental audits: run `pip list` in the local virtual environment, if present

    Environmental audits: run `pip list` in the local virtual environment, if present

    This is a follow-on to #450.

    #451 "resolves" the issue by emitting a warning on the current ambiguity, but we can do better than that: instead of just warning, we can use ${VIRTUAL_ENV}/bin/python as pip_api's Python base, which in turn will allow globally installed copies of pip-audit to audit local virtual environments (as users might intuitively expect).

    To do that, we'll need to re-think how we use pip_api, and possibly upstream some changes:

    • We can't rely on pip_api.PIP_VERSION, since it's computed eagerly from whatever pip was found at module import time. Maybe we should just use pip_api.version().
    • It might make sense to change how pip_api currently supports overrides -- PIPAPI_PYTHON_LOCATION in the environment could probably be replaced with something more ergonomic.
    enhancement 
    opened by woodruffw 0
  • Post-mortem: regression in 2.4.9

    Post-mortem: regression in 2.4.9

    We had a regression in 2.4.9, caused by a type error: #439.

    In particular, we refactored a file-like type into a pathlib.Path, but forgot to update one of the use sites. This should have been caught by mypy, so I'm filing this post-mortem to figure out why it wasn't (and what we can do to fix that).

    opened by woodruffw 5
  • Hash checking: respect third party indices

    Hash checking: respect third party indices

    Is your feature request related to a problem? Please describe. Yes.

    Currently if you are using a privately hosted repo and building your own wheels for various projects, you cannot use pip-audit -s pypi, while only listing the hashes of your privately hosted wheels. This leaves us with 2 options. Adding the upstream digest hash ( that we dont use ) to our hash list for each package. Or using -s osv.

    Currently we are using osv. And though I am unsure if this is inferior. We would like to use the default where we can.

    Describe the solution you'd like

    Ideally - https://github.com/pypa/pip-audit/blob/76e4fa4a0ed005f543701d4b60c02c75c5b539d2/pip_audit/_service/pypi.py#L89

    Would have logic have a flag around it to disable hash checking for situations like this.

    Describe alternatives you've considered

    As mentioned above using -s osv

    Additional context

    Example of current failure:

    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/adal/1.2.2/json HTTP/1.1" 200 1128
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/adal/1.2.2/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/aenum/3.1.0/json" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/aenum/3.1.0/json HTTP/1.1" 200 2235
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/aenum/3.1.0/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/aiobotocore/1.2.2/json" in the cache
    DEBUG:cachecontrol.controller:No cache entry available
    DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/aiobotocore/1.2.2/json HTTP/1.1" 200 5390
    DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/aiobotocore/1.2.2/json"
    DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
    DEBUG:cachecontrol.controller:Caching due to etag
    Traceback (most recent call last):
      File "/Users/carl.gill/.pyenv/versions/3.7.3/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_cli.py", line 439, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_audit.py", line 67, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_service/interface.py", line 156, in query_all
        yield self.query(spec)
      File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_service/pypi.py", line 101, in query
        f"Mismatched hash for {spec.canonical_name} ({spec.version}): listed "
    pip_audit._service.interface.ServiceError: Mismatched hash for aiobotocore (1.2.2): listed 290833e77f3992cf947279106e43f97dc66f72731cf6818d89a171295bda79ba of type sha256 could not be found in PyPI releases```
    bug question component:dep-sources 
    opened by carl-armis 12
  • Cannot handle =" in requirements.txt">

    Cannot handle ">=" in requirements.txt

    The vast vast majority of Python packages use >= in their requirements.txt not ==. But pip-audit is not flagging vulnerabilities when >= used.

    E.g., take this example requirements.txt:

    lxml>=4.5.1

    lxml 4.5.1 contains a vulnerability but is not flagged by pip-audit. Only flagged if >= replaced with ==

    discussion 
    opened by ValueRaider 7
  • `--requirement -` does not work: `No such file or directory: '<stdin>'`

    `--requirement -` does not work: `No such file or directory: ''`

    Bug description

    When running pip-audit with --requirement - does not read the requirements from stdin like --requirement /dev/stdin does but instead raises an exception pip_requirements_parser.InstallationError: Could not open requirements file: <stdin>|n[Errno 2] No such file or directory: '<stdin>'.

    Reproduction steps

    echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement -
    

    Expected behavior

    I expect this to do the same as:

    echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement /dev/stdin
    

    i.e.

    $ echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement /dev/stdin
    ⚠️  pip-audit is already on your PATH and installed at /home/iwana/.local/bin/pip-audit. Downloading and running anyway.
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
    Found 1 known vulnerability in 1 package
    Name Version ID                  Fix Versions
    ---- ------- ------------------- ------------
    cleo 1.0.0a5 GHSA-2p9h-ccw7-33gf
    

    Screenshots and logs

    instead this is what happens:

    $ echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement -
    ⚠️  pip-audit is already on your PATH and installed at /home/iwana/.local/bin/pip-audit. Downloading and running anyway.
    Traceback (most recent call last):
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1577, in get_file_content
        with open(filename, "rb") as f:
    FileNotFoundError: [Errno 2] No such file or directory: '<stdin>'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/bin/pip-audit", line 8, in <module>
        sys.exit(audit())
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_cli.py", line 434, in audit
        for (spec, vulns) in auditor.audit(source):
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_audit.py", line 66, in audit
        for dep, vulns in self._service.query_all(specs):
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_service/interface.py", line 155, in query_all
        for spec in specs:
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 82, in collect
        rf = RequirementsFile.from_file(filename)
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 233, in from_file
        for parsed in cls.parse(
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 277, in parse
        for parsed in parse_requirements(
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1152, in parse_requirements
        for parsed_line in parser.parse(
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1329, in parse
        yield from self._parse_and_recurse(
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1351, in _parse_and_recurse
        for line in self._parse_file(filename=filename, is_constraint=is_constraint):
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1404, in _parse_file
        content = get_file_content(filename)
      File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1580, in get_file_content
        raise InstallationError(
    pip_requirements_parser.InstallationError: Could not open requirements file: <stdin>|n[Errno 2] No such file or directory: '<stdin>'
    

    Platform information

    • OS name and version: Fedora Linux 36 (Xfce)
    • pip-audit version (pip-audit -V): 2.4.6
    • Python version (python -V or python3 -V): 3.10.8
    • pip version (pip -V or pip3 -V): Using pipx, so none.

    Additional context

    I'm trying to use pip-audit with poetry, and to do this I run:

    poetry export --without-hashes --with dev --format requirements.txt | \
      poetry run python -m pip_audit --requirement /dev/stdin --no-deps --strict --desc on
    

    However this will probably not work great on windows, and I would prefer a less platform dependent way of specifying stdin, like --requirement -.

    enhancement upstream 
    opened by aucampia 2
  • Add support for Poetry

    Add support for Poetry

    If the project path is specified in CLI, read dependencies from poetry.lock file (if available).

    I'm quite happy with collect. Since this is a lock file, we can safely skip dependency resolution, as we do for pip source. And the format of the file is quite simple, so we don't depend on poetry for reading it. Cool!

    Autofix is not supported yet because poetry needs hashes to be updated, see review comments.

    I checked the change manually on a few quite big poetry projects, works like a charm.

    Close #84

    opened by orsinium 2
Releases(v2.4.12)
Owner
Trail of Bits
More code: binary lifters @lifting-bits, blockchain @crytic
Trail of Bits
Run with one command grafana, prometheus, and a python script to collect and display cryptocurrency prices and track your wallet balance.

CryptoWatch Track your favorite crypto coin price and your wallet balance. Install Create .env: ADMIN_USER=admin ADMIN_PASSWORD=admin Configure you

Rafael Zimmermann 13 Dec 13, 2022
Bit is Python's fastest Bitcoin library and was designed from the beginning to feel intuitive, be effortless to use, and have readable source code.

Bit is Python's fastest Bitcoin library and was designed from the beginning to feel intuitive, be effortless to use, and have readable source code.

Ofek Lev 1.1k Jan 2, 2023
A Python library to wrap age and minisign to provide key management, encryption/decryption and signing/verification functionality.

A Python library to wrap age and minisign to provide key management, encryption/decryption and signing/verification functionality.

Vinay Sajip 3 Feb 1, 2022
Alpkunt 9 Sep 9, 2022
Python-RSA is a pure-Python RSA implementation.

Pure Python RSA implementation Python-RSA is a pure-Python RSA implementation. It supports encryption and decryption, signing and verifying signatures

Sybren A. Stüvel 418 Jan 4, 2023
Python binding to the Networking and Cryptography (NaCl) library

PyNaCl: Python binding to the libsodium library PyNaCl is a Python binding to libsodium, which is a fork of the Networking and Cryptography library. T

Python Cryptographic Authority 941 Jan 4, 2023
cryptography is a package designed to expose cryptographic primitives and recipes to Python developers.

pyca/cryptography cryptography is a package which provides cryptographic recipes and primitives to Python developers. Our goal is for it to be your "c

Python Cryptographic Authority 5.2k Dec 30, 2022
Python ASN.1 library with a focus on performance and a pythonic API

asn1crypto A fast, pure Python library for parsing and serializing ASN.1 structures. Features Why Another Python ASN.1 Library? Related Crypto Librari

Will Bond 282 Dec 11, 2022
Freqtrade is a free and open source crypto trading bot written in Python

Freqtrade is a free and open source crypto trading bot written in Python. It is designed to support all major exchanges and be controlled via Telegram. It contains backtesting, plotting and money management tools as well as strategy optimization by machine learning.

null 20.2k Jan 7, 2023
This python module can analyse cryptocurrency news for any number of coins given and return a sentiment. Can be easily integrated with a Trading bot to keep an eye on the news.

Python script that analyses news headline or body sentiment and returns the overall media sentiment of any given coin. It can take multiple coins an

null 185 Dec 22, 2022
Mysterium the first tool which permits you to retrieve the most part of a Python code even the .py or .pyc was extracted from an executable file, even it is encrypted with every existing encryptage. Mysterium don't make any difference between encrypted and non encrypted files, it can retrieve code from Pyarmor or .pyc files.

Mysterium the first tool which permits you to retrieve the most part of a Python code even the .py or .pyc was extracted from an executable file, even it is encrypted with every existing encryptage. Mysterium don't make any difference between encrypted and non encrypted files, it can retrieve code from Pyarmor or .pyc files.

Venax 116 Dec 21, 2022
A tool used to encrypt Python scripts version < 2.7 and version < 3.9

A tool used to encrypt Python scripts version < 2.7 and version < 3.9

Fajar Kim 1 Dec 14, 2021
Python program that handles the creation, encryption and storage of log/journal files. Kinda works like a diary of sorts.

LucaSoft J.O.U.R.N.A.L The J.O.U.R.N.A.L (Just anOther User Redaction & Navigation Assistant by Lucaspec72) is a Python program that handles the creat

Lucaspec72 8 Oct 27, 2021
A python tool to track prices of various cryptocurrencies and alert

CryptoPriceTracker This is a tool to track prices of various cryptocurrencies and alert the user once the user defined maximum & minimum target is rea

null 1 Oct 1, 2021
This is a basic encryption and decryption program made in python.

A basic encryption and decryption program to visualize how the process of protecting data works.

Junaid Chaudhry 5 Nov 15, 2021
A Python Tool to encrypt all types of files using AES and XOR Algorithm.

DataShield This project intends to protect user’s data, it stores files in encrypted format in device provided the passcode and path of the file. AES

ADITYA SHINDE 4 Dec 20, 2021
Python Script for signingn LetsEncrypt certificate with certbot, and update them into Fortigate

Python Script for signingn LetsEncrypt certificate with certbot, and update them into Fortigate (to be used into the WEB VPN or Load Balancer certificate)

null 6 Jan 3, 2023
This is simple Blockchain ,miner and wallet to send crypto using python

pythonBlockchain-SImple This is simple Blockchain ,miner and wallet to send crypto using python It is simple Blocchain so it can only dobasic work usi

null 3 Nov 22, 2022
A short code in python, Enchpyter, is able to encrypt and decrypt words as you determine, of course

Enchpyter Enchpyter is a program do encrypt and decrypt any word you want (just letters). You enter how many letters jumps and write the word, so, the

João Assalim 2 Dec 9, 2021