pipreqs - Generate pip requirements.txt file based on imports of any project. Looking for maintainers to move this project forward.

Related tags

pipreqs
Overview

pipreqs - Generate requirements.txt file for any project based on imports

Installation

pip install pipreqs

Usage

Usage:
    pipreqs [options] <path>

Options:
    --use-local           Use ONLY local package info instead of querying PyPI
    --pypi-server <url>   Use custom PyPi server
    --proxy <url>         Use Proxy, parameter will be passed to requests library. You can also just set the
                          environments parameter in your terminal:
                          $ export HTTP_PROXY="http://10.10.1.10:3128"
                          $ export HTTPS_PROXY="https://10.10.1.10:1080"
    --debug               Print debug information
    --ignore <dirs>...    Ignore extra directories
    --encoding <charset>  Use encoding parameter for file open
    --savepath <file>     Save the list of requirements in the given file
    --print               Output the list of requirements in the standard output
    --force               Overwrite existing requirements.txt
    --diff <file>         Compare modules in requirements.txt to project imports.
    --clean <file>        Clean up requirements.txt by removing modules that are not imported in project.
    --no-pin              Omit version of output packages.

Example

$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt

Contents of requirements.txt

wheel==0.23.0
Yarg==0.1.9
docopt==0.6.2

Why not pip freeze?

  • pip freeze only saves the packages that are installed with pip install in your environment.
  • pip freeze saves all packages in the environment including those that you don't use in your current project. (if you don't have virtualenv)
  • and sometimes you just need to create requirements.txt for a new project without installing modules.
Issues
  • Error while creating requirements.txt

    Error while creating requirements.txt

    Hello!

    C:\Users\ipetrash\Desktop\PyScripts\hideme.ru-proxy-list>pipreqs C:\Users\ipetrash\Desktop\PyScripts\hideme.ru-proxy-list
    Traceback (most recent call last):
      File "C:\Python34\lib\runpy.py", line 170, in _run_module_as_main
        "__main__", mod_spec)
      File "C:\Python34\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "C:\Python34\Scripts\pipreqs.exe\__main__.py", line 9, in <module>
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 178, in main
        init(args)
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 157, in init
        local = get_import_local(candidates)
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 112, in get_import_local
        local = get_locally_installed_packages()
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 105, in get_locally_installed_packages
        'version': package[1].replace(".dist", ""),
    IndexError: list index out of range
    

    Used for: https://github.com/gil9red/hideme.ru-proxy-list

    Sincerely yours, gil9red

    bug 
    opened by gil9red 23
  • New Option for Dynamic Versioning, e.g. wheel==0.23.*, wheel==0.* or wheel

    New Option for Dynamic Versioning, e.g. wheel==0.23.*, wheel==0.* or wheel

    Added new option "dynamic", for dynamic versioning.

    Example: wheel==0.23.0

    There will be 3 possible args for this option, micro, minor or all. The result in requirements.txt will be as below.

    • wheel==0.23.* (micro)
    • wheel==0.* (minor)
    • wheel (all)

    This will enable more targeted installations, esp if one is only looking at bug fixes, security patches or minor/micro version updates, or simply just install most recent updates.

    --no-pin option is merged within this new option as the "all" arg.

    @jcch94

    opened by mapattacker 13
  • added support for >= in addition to ==

    added support for >= in addition to ==

    using flag --gt (gt = greater than)

    opened by talperetz 10
  • Pipreqs missing some requirements

    Pipreqs missing some requirements

    I develop this project: https://github.com/tuwien-musicir/rp_extract

    As you can see from the imports it uses a lot of libraries (numpy, scipy, matplotlib, argparse, unicsv, cPickle, sklearn, ...) - many of which are missed by pipreqs:

    pipreqs --debug --force . DEBUG: Found packages: set(['argparse', 'cPickle', 'uuid', 'tempfile', 'numpy', 'pylab', 'matplotlib', 'wave', 'sys', 'subprocess', 'os', 'sklearn']) DEBUG: Found imports: matplotlib, numpy, matplotlib, scikit_learn DEBUG: Getting packages information from Local/PyPI INFO: Starting new HTTPS connection (1): pypi.python.org DEBUG: Setting read timeout to None DEBUG: "GET /pypi/numpy/json HTTP/1.1" 200 65657 INFO: Starting new HTTPS connection (1): pypi.python.org DEBUG: Setting read timeout to None DEBUG: "GET /pypi/scikit_learn/json HTTP/1.1" 301 0 DEBUG: Setting read timeout to None DEBUG: "GET /pypi/scikit-learn/json HTTP/1.1" 200 102410 DEBUG: Writing 4 requirements: matplotlib, matplotlib, numpy, scikit_learn to ./requirements.txt INFO: Successfully saved requirements file in ./requirements.txt [email protected]:~/rp_extract$ cat requirements.txt matplotlib == 1.3.1.egg matplotlib == 1.3.1.egg numpy == 1.10.1 scikit_learn == 0.17

    Why does it miss many of the library requirements? (some of them are even mentioned in the debug output, some are not, e.g. scipy, unicsv) (and matplotlib on the other hand is put twice into the requirements.txt)

    bug 
    opened by audiofeature 10
  • Error executing newly installed pipreqs

    Error executing newly installed pipreqs

    I installed pipreqs using pip install pipreqs within my local virtualenv and then executed it, got this error:

    ~/code % pipreqs web
    Looking for imports
    Traceback (most recent call last):
      File "/home/satiani/code/web/venv/bin/pipreqs", line 8, in <module>
        load_entry_point('pipreqs==0.1.8', 'console_scripts', 'pipreqs')()
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 109, in main
        init(args)
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 92, in init
        imports = get_all_imports(args['<path>'])
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 62, in get_all_imports
        with open(os.path.join(os.path.dirname(__file__), "stdlib"), "r") as f:
    IOError: [Errno 2] No such file or directory: '/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/stdlib'
    zsh: exit 1     pipreqs web
    
    bug 
    opened by satiani 10
  • Ordering imports by name

    Ordering imports by name

    Description:

    • Order the imports generated by name

    Tests:

    • Tests support both 2 and 3
    opened by annp89 9
  • Difference between package name and import name makes problem

    Difference between package name and import name makes problem

    For example, to import beautifulsoup4 package, it should be import bs4. So pipreqs cannot find the beautifulsoup4 package from pypi server. I don't have an idea to resolve this problem... :(

    bug 
    opened by littmus 9
  • pipreqs not capturing all imports - recurly

    pipreqs not capturing all imports - recurly

    I am having issues capturing Recurly lib. pip install recurly

    Leverage it with a python file via import

    https://pypi.org/project/recurly

    opened by smittysmee 8
  • whitelines at the beginning of the file

    whitelines at the beginning of the file

    pipreqs fails on python files which have blank lines at the beginning of the file.

    System:

    • Python 3.6.8
    • 16.04.1-Ubuntu
    • pipreqs 0.4.10

    Stacktrace:

    ERROR: Failed on file: /home/test/file.py
    Traceback (most recent call last):
      File "/usr/local/bin/pipreqs", line 10, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 470, in main
        init(args)
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 409, in init
        follow_links=follow_links)
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 138, in get_all_imports
        raise exc
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 124, in get_all_imports
        tree = ast.parse(contents)
      File "/usr/lib/python3.6/ast.py", line 35, in parse
        return compile(source, filename, mode, PyCF_ONLY_AST)
      File "<unknown>", line 1
        
        ^
    SyntaxError: invalid character in identifier
    
    opened by EliasVansteenkiste 8
  • Exception when 'import' is used in comments

    Exception when 'import' is used in comments

    A freshly installed pipreqs (in a venv) crashes on pylint source (in a venv in my case)

    Offending line is: https://bitbucket.org/logilab/pylint/src/cc5990f6a12a467d09567bfc1b210ba999f6cb13/pylint/test/functional/unused_import.py?at=default#cl-3

    ~/src/c3 $ pipreqs .
    Traceback (most recent call last):
      File "/home/koert/src/c3/.ve2/bin/pipreqs", line 11, in <module>
        sys.exit(main())
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 167, in main
        init(args)
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 139, in init
        candidates = get_all_imports(args['<path>'])
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 48, in get_all_imports
        res = map(get_name_without_alias, item.split(","))
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 130, in get_name_without_alias
        name = REGEXP[0].match(name.strip()).groups(0)[0]
    AttributeError: 'NoneType' object has no attribute 'groups'
    ~/src/c3 $ pip freeze
    docopt==0.6.2
    pipreqs==0.2.4
    requests==2.7.0
    yarg==0.1.9
    
    bug 
    opened by kvdveer 8
  • SyntaxError: invalid non-printable character

    SyntaxError: invalid non-printable character

    Code can not contain characters with diacritics (á,é,ô,ň), else pipreqs breaks.

    opened by oguh43 1
  • probably not necessary inserting of the auto_mix_prep library to the requirements.txt by pipreqs

    probably not necessary inserting of the auto_mix_prep library to the requirements.txt by pipreqs

    Hello together, thank you for your nice little tool. It worked pretty well. After the usage, I just fund the auto_mix_prep library in the requirements.txt which I did not even have installed while my repository was working correctly in my environment.

    Then I had a look in your code and found the following two lines in the "mapping" - file and found the following rows 1046 and 1047:

        src:atlas
        src:auto_mix_prep
    

    After a look at the pipreqs.py it seems to me like some import from the source folder inside of my repository is causing to add auto_mix_prep to the requirements.txt py pipreqs.

    The deletion of the row "src:auto_mix_prep" will probably cause to add the atlas-library.

    Is any of these rows necessary? Especially since src is a common name for code inside a repository. These rows will cause the auto_mix_prep to be installed uselessly in really a lot of virtual environments if I understood it correctly.

    I would propose to delete these rows if there is no real reason for them.

    opened by ThomasRegier 0
  • Small change to make debugging file parse errors easier 2

    Small change to make debugging file parse errors easier 2

    Now when ast.parse() encounters a file that cannot be parsed due to syntax errors it will report that filename in it's traceback.

    opened by abrahamw88 2
  • Not adding all required packages that are used in the Django project

    Not adding all required packages that are used in the Django project

    In my project along with Django package, I have installed other packages ie. django-crispy-forms and so on. However, when I use the pipreqs only the Django package is added but not other required packages.

    bug mapping 
    opened by kushal-h 3
  • Flake8 code quality #274

    Flake8 code quality #274

    Updated code formatting & long line break changes Added warning ignore comments at unused imports, duplicate imports, multiple imports Added line spaces after function definition

    opened by kuldeepkhatke 2
  • Tests do not respect flake8

    Tests do not respect flake8

    since we tested flake8 compliance with tox, the tests where never validated for flake8 violations

    good first issue 
    opened by alan-barzilay 4
  • Error if script contains Umlauts (like ä,ö,ü)

    Error if script contains Umlauts (like ä,ö,ü)

    pipreqs trows Error when script contains umlauts: Stack trace:

    ERROR: Failed on file: P:\PROJECTS\Dataplot_Preislistendownload\Dataplot_Preisliste.py
    Traceback (most recent call last):
      File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "c:\python39\lib\runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "C:\Python39\Scripts\pipreqs.exe\__main__.py", line 7, in <module>
      File "c:\python39\lib\site-packages\pipreqs\pipreqs.py", line 470, in main
        init(args)
      File "c:\python39\lib\site-packages\pipreqs\pipreqs.py", line 406, in init
        candidates = get_all_imports(input_path,
      File "c:\python39\lib\site-packages\pipreqs\pipreqs.py", line 138, in get_all_imports
        raise exc
      File "c:\python39\lib\site-packages\pipreqs\pipreqs.py", line 124, in get_all_imports
        tree = ast.parse(contents)
      File "c:\python39\lib\ast.py", line 50, in parse
        return compile(source, filename, mode, flags,
      File "<unknown>", line 89
        df = df.assign(Wiederverkäufer = round_col(vk - (vk /WVfaktor),2))
                                  ^
    

    attached affecting Script (please change to *.py :-) ) Dataplot_Preislist.txt

    bug 
    opened by sgaechter 0
  • pipreqs uses the wrong package

    pipreqs uses the wrong package

    i don't know how to describe the problem, but pipreqs always replaces ffmpeg-python with ffmpeg. any plan how to fix that?

    opened by frankenstein91 0
  • Enable ignore erros

    Enable ignore erros

    Allowing users to pass "--ignore-errors" and ignore files that cannot be parsed due to syntax errors. Passed tests locally.

    opened by avnaf 0
  • Fix Issue #227

    Fix Issue #227

    Fixed SSL issues encountered behind enterprise proxies. I encourage contributors to write the test cases which can test this functionality.

    opened by devashish-gaikwad 0
Releases(v0.4.10)
Owner
Vadim Kravcenko
CTO @ Mindnow AG
Vadim Kravcenko
:package: :fire: Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.

THE PROJECT IS ARCHIVED Forks: https://github.com/orsinium/forks DepHell -- project management for Python. Why it is better than all other tools: Form

DepHell 1.7k Oct 22, 2021
An installation and dependency system for Python

Pyflow Simple is better than complex - The Zen of Python Pyflow streamlines working with Python projects and files. It's an easy-to-use CLI app with a

David O'Connor 904 Oct 19, 2021
A PyPI mirror client according to PEP 381 http://www.python.org/dev/peps/pep-0381/

This is a PyPI mirror client according to PEP 381 + PEP 503 http://www.python.org/dev/peps/pep-0381/. bandersnatch >=4.0 supports Linux, MacOSX + Wind

Python Packaging Authority 263 Oct 17, 2021
A PyPI mirror client according to PEP 381 http://www.python.org/dev/peps/pep-0381/

This is a PyPI mirror client according to PEP 381 + PEP 503 http://www.python.org/dev/peps/pep-0381/. bandersnatch >=4.0 supports Linux, MacOSX + Wind

Python Packaging Authority 262 Oct 11, 2021
Conan - The open-source C/C++ package manager

Conan Decentralized, open-source (MIT), C/C++ package manager. Homepage: https://conan.io/ Github: https://github.com/conan-io/conan Docs: https://doc

Conan.io 5.3k Oct 22, 2021
pip-run - dynamic dependency loader for Python

pip-run provides on-demand temporary package installation for a single interpreter run. It replaces this series of commands (or their Windows equivale

Jason R. Coombs 61 Oct 2, 2021
A Poetry plugin for dynamically extracting the package version.

Poetry Version Plugin A Poetry plugin for dynamically extracting the package version. It can read the version from a file __init__.py with: # __init__

Sebastián Ramírez 148 Oct 9, 2021
Python Development Workflow for Humans.

Pipenv: Python Development Workflow for Humans [ ~ Dependency Scanning by PyUp.io ~ ] Pipenv is a tool that aims to bring the best of all packaging wo

Python Packaging Authority 22.4k Oct 24, 2021
The Fast Cross-Platform Package Manager

The Fast Cross-Platform Package Manager part of mamba-org Package Manager mamba Package Server quetz Package Builder boa mamba Mamba is a reimplementa

Mamba 2.3k Oct 24, 2021
The Python package installer

pip - The Python Package Installer pip is the package installer for Python. You can use pip to install packages from the Python Package Index and othe

Python Packaging Authority 7.5k Oct 23, 2021
A PDM plugin that packs your packages into a zipapp

pdm-packer A PDM plugin that packs your packages into a zipapp Requirements pdm-packer requires Python >=3.7 Installation If you have installed PDM wi

Frost Ming 10 Sep 23, 2021
Install and Run Python Applications in Isolated Environments

pipx — Install and Run Python Applications in Isolated Environments Documentation: https://pipxproject.github.io/pipx/ Source Code: https://github.com

null 4.2k Oct 22, 2021
Python dependency management and packaging made easy.

Poetry: Dependency Management for Python Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right sta

Poetry 16.8k Oct 24, 2021
Python dependency management and packaging made easy.

Poetry: Dependency Management for Python Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right sta

Poetry 16.8k Oct 22, 2021
For when Poetry just doesn't work.

Ballad For when Poetry just doesn't work. Have you tried setting up Poetry, but something doesn't work? Maybe you're... Trying to implement Github Act

BD103 4 Sep 22, 2021
Solaris IPS: Image Packaging System

Solaris Image Packaging System Introduction The image packaging system (IPS) is a software delivery system with interaction with a network repository

Oracle 47 Sep 17, 2021
Package manager based on libdnf and libsolv. Replaces YUM.

Dandified YUM Dandified YUM (DNF) is the next upcoming major version of YUM. It does package management using RPM, libsolv and hawkey libraries. For m

null 911 Oct 17, 2021
local pypi server (custom packages and auto-mirroring of pypi)

localshop A PyPI server which automatically proxies and mirrors PyPI packages based upon packages requested. It has support for multiple indexes and t

Michael van Tellingen 372 Aug 29, 2021
[DEPRECATED] YUM package manager

⛔ This project is deprecated. Please use DNF, the successor of YUM. YUM Yum is an automatic updater and installer for rpm-based systems. Included prog

null 94 Aug 5, 2021