C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Overview

logo

https://travis-ci.org/rakhimov/cppdep.svg?branch=master 'Build status' Code Health

cppdep performs dependency analysis among components/packages/package groups of a large C/C++ project. This is a rewrite of dep_utils(adep/cdep/ldep), which is provided by John Lakos' book "Large-Scale C++ Software Design", Addison Wesley (1996).

Limitations

  • Indirect extern declarations of global variables or functions instead of including the proper component header with the declarations.
  • Embedded dynamic dependencies, such as dynamic loading and configurable internal services.
  • Preprocessing or macro expansion is not performed. Dependency inclusion via preprocessor meta-programming is not handled.
  • Dependency exclusion with C style multi-line comments or macros is not respected.

Requirements

  1. Python 2.7 or 3.4+
  2. NetworkX
  3. pydot
  4. pydotplus
  5. PyYAML
  6. PyKwalify 1.6.0+

The dependencies can be installed with pip.

$ sudo pip install -r requirements.txt

Installation

From the source:

$ ./setup.py install

The latest stable release from PyPi:

$ pip install cppdep

Usage

Create a configuration file that describes the project for analysis. config_schema.yml is given for guidance.

In the root directory of the project with the configuration file, run the following command to generate dependency analysis reports and graphs.

$ cppdep -c /path/to/config/file

More documentation and example configurations can be found in project wiki.

Acknowledgments

  • John Lakos for inventing the analysis and providing dep_utils.
  • Zhichang Yu for rewriting dep_utils into Python.
Comments
  • Behavior specification for anomalous conflicting component files

    Behavior specification for anomalous conflicting component files

    Anomalous (rare, error) case of having "component.c" and "component.cc" or "component.h" and "component.hpp" at the same time trying to define the same component. In other words, files with the same basenames but different extensions in header and/or source groups.

    opened by rakhimov 1
  • Missing 'pydot' dependency fails on networkx nx_pydot

    Missing 'pydot' dependency fails on networkx nx_pydot

    Traceback (most recent call last):
      File "/.../python3env/bin/cppdep", line 11, in <module>
        sys.exit(main())
      File "/.../python3env/lib/python3.5/site-packages/cppdep/__main__.py", line 59, in main
        analysis.analyze(printer, args)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 797, in analyze
        lambda x: isinstance(x, PackageGroup)))
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 779, in _analyze
        digraph.write_dot(graph_name)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/graph.py", line 273, in write_dot
        write_dot(self.digraph, file_basename + '.dot')
      File "<decorator-gen-416>", line 2, in write_dot
      File "/.../python3env/lib/python3.5/site-packages/networkx/utils/decorators.py", line 224, in _open_file
        result = func(*new_args, **kwargs)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 54, in write_dot
        P = to_pydot(G)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 199, in to_pydot
        pydot = _import_pydot()
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 348, in _import_pydot
        import pydot
    
    bug 
    opened by rakhimov 0
  • Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    If component/project names contain paths as identifiers, the path separator on Windows \ followed by 'U' or 'u' is interpreted as a Unicode literal by pydotplus plotting on Windows with python 2.7.

    This problem does not show up on python 3.

    For the sake of consistency in the report (stable report) and testing, use Unix path separator in Ids.

    bug 
    opened by rakhimov 0
  • Handle 'ipp' template implementation/source files

    Handle 'ipp' template implementation/source files

    Even though templates are only in headers, ipp files could be considered as implementation pair of the interface header. This occurs, for example, in Boost. #29.

    opened by rakhimov 0
  • Redundant 'ldep' printing of cumulative dependencies

    Redundant 'ldep' printing of cumulative dependencies

    Unlike the original ldep, the cppdep is printing the whole cumulative dependencies; that is, the all link time dependencies are printed. This information is visually provided in the graph, and implicitly provided in the original ldep '-l | -L' flags.

    It seems like the current implementation of the cumulative dependency printing is geared towards debugging rather than analysis report.

    Fix this printing by implementing the original behavior of ldep.

    bug 
    opened by rakhimov 0
  • Pairing header and implementation files in different locations

    Pairing header and implementation files in different locations

    Header and implementation files can be located in different directories, e.g., headers in include and implementation in src. The current implementation assumes the same location.

    bug enhancement 
    opened by rakhimov 0
  • Deduce external packages from the include directives w/o filesystem search

    Deduce external packages from the include directives w/o filesystem search

    The package can be deduced from the include directives following the pattern: "<package/header>", e.g., "<boost/any.hpp>". This approach will avoid the relatively expensive lookup of the header from the system. Whether such external header actually exists on the system or not is irrelevant to the dependency analysis.

    This heuristic would make the configuration simpler and more robust for cross-platform work.

    In addition, the standard library headers can be "hard-coded" into the script since these are the most likely to be used/searched by the source files.

    There's a small chance of false positives if the project under analysis happens not to conform to this convention and invents its own packages with the same names as external ones, e.g., boost, qt, libxml.

    enhancement 
    opened by rakhimov 0
  • Double counting of common components in CCD

    Double counting of common components in CCD

    The current implementation overcounts link time dependencies with common components deep in the dependency graph. Consider: A->B->C, A->D->C. CCD(A) must be 4 (4 object files in total.) The current approach overcounts the common object C, so CCD(A) ends up being 5.

    bug 
    opened by rakhimov 0
  • Extended definition for 'Component'

    Extended definition for 'Component'

    Even though John Lakos defined a component as a pair of h and c files, C++ can have template only components residing only in header files (e.g., STL/Boost/etc.). Moreover, some header-only components may contain only inline functions or macros without any need for an implmentation file (e.g., inline math, Boost PPL). For these reason, unpaired header files can be counted as components by default.

    In addition, the implementation file containing the main function of an application could be considered as a component as well (an entry point).

    bug enhancement 
    opened by rakhimov 0
  • Incorrect dependency processing with file basenames

    Incorrect dependency processing with file basenames

    The current implementation uses the basename of the included files as the key in dependency search. This approach ignores the current location of the dependent files and the language include rules, i.e. <header.h> vs. "header.h". Moreover, it results in name conflicts for files that may belong to completely separate projects or packages.

    bug 
    opened by rakhimov 0
  • Compatibility fixes

    Compatibility fixes

    This PR contains some fixes that I needed to get the cppdep tool running in my recent python 3.8 environment. Also handling of errors in the yaml file was adapted a little bit to get more helpful error messages

    opened by jsinge 0
  • Missing graph module

    Missing graph module

    I've just installed this project, tried to run it but I get this error message.

    Traceback (most recent call last):
      File "cppdep.py", line 36, in <module>
        from .graph import Graph
    ModuleNotFoundError: No module named '__main__.graph'; '__main__' is not a package
    
    bug help wanted question 
    opened by nbourre 2
  • Python 3.6 os.path.commonprefix expects List

    Python 3.6 os.path.commonprefix expects List

      File "/.../cppdep/cppdep.py", line 92, in path_common
        path = os.path.commonprefix(paths)
      File "/home/olzhas/temp/pyenv/lib/python3.6/genericpath.py", line 76, in commonprefix
        if not isinstance(m[0], (list, tuple)):
    TypeError: 'set' object does not support indexing
    
    bug 
    opened by rakhimov 1
  • Include Wrangler Analysis

    Include Wrangler Analysis

    Incorporation of include-wrangler analysis

    • header file cost
    • include cost
    • translation unit cost
    • other features

    In collaboration with include-wrangler author @lukedodd

    enhancement 
    opened by rakhimov 0
  • Options to disable/enable warnings

    Options to disable/enable warnings

    • [ ] Duplicate include
    • [ ] Redundant (transitive from the component header) include
    • [ ] Missing a component header (incomplete component)
    • [ ] Failure to locate a header from an include directive
    enhancement 
    opened by rakhimov 0
Releases(0.2.0)
  • 0.2.0(Feb 3, 2017)

    Added

    • Pairing header and implementation files in different locations (#19)
    • Handle 'ipp' template implementation source files (#31)
    • Behavior specification for anomalous conflicting component files (#27)
    • Implement ignore/exclude paths (#23)
    • Accept glob pattern for source paths (#36)
    • Project wiki pages
    • Regex pattern based include directive classification (#22)
    • Deduce external packages from the include directive w/o filesystem search (#18)
    • Handle header files w/o extensions (Boost/STL/Qt/etc.) (#32)
    • Use POSIX path separator in component names (for cross-platform report stability)
    • Configuration file validation against the schema (with PyKwalify)

    Changed

    • pytest instead of nose
    • YAML configuration files instead of XML (#24)

    Removed

    • Implicit single-path alias Package construction

    Fixed

    • Exception leaks out of main()
    • Unicode Escape Error on graph dot on Windows with Python 2.7 (#35)
    • Python3 UnicodeDecodeError for 'utf-8' in source files (#30)
    • Logging: Type Error: not all arguments converted during string formatting (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 6, 2017)

    Added

    • The original ldep '-l|-L' options to print dependencies (#20)
    • '-o' to print reports into a file
    • Warn about duplicate and redundant includes (#13)
    • Extended definition for 'Component' (#7)
    • PEP-257 conformance (#2)
    • PEP-8 conformance (#1)
    • Python 3 support
    • PyPI package
    • XML configuration example and RNG schema
    • Travis CI (Linux, OS X) and AppVeyor CI (Windows) setups

    Changed

    • Differentiate 'paths' into source, include, and alias.
    • Print warnings to stderr instead of stdout (#12)
    • Report Component levels instead of Graph layers (#9)
    • Refactor the procedural design into the object-oriented design (#4)
    • Change '-f' flag into '-c' flag
    • Replace optparse with argparse
    • XML configuration file format

    Removed

    • Redundant printing a list of cumulative dependencies (#20)
    • Indirect missing-header include warnings
    • Global cross-package and cross-package-group component dependency analysis
    • 'details-of-components/--debug' verbosity
    • dot2any.py helper script
    • Manual profiling code (use pyvmmonitor instead)
    • Manual testing code (automated with nosetest)

    Fixed

    • Level 0 External components missing from the report and graph (#21)
    • Incorrect dependency processing with file basenames (#6)
    • Wrong level calculation for cycles (#8)
    • Double counting of common components in CCD calculations (#11)
    • Missing cycles from the Dot graph (#10)
    • Outdated networkx API usage
    Source code(tar.gz)
    Source code(zip)
Owner
Olzhas Rakhimov
Olzhas Rakhimov
Dependency Combobulator is an Open-Source, modular and extensible framework to detect and prevent dependency confusion leakage and potential attacks.

Dependency Combobulator Dependency Combobulator is an Open-Source, modular and extensible framework to detect and prevent dependency confusion leakage

Apiiro 84 Dec 23, 2022
DepFine Is a tool to find the unregistered dependency based on dependency confusion valunerablility and lead to RCE

DepFine DepFine Is a tool to find the unregistered dependency based on dependency confusion valunerablility and lead to RCE Installation: You Can inst

Hossam mesbah 14 Nov 11, 2022
A Python implementation of John Gruber’s Markdown with Extension support.

Python-Markdown This is a Python implementation of John Gruber's Markdown. It is almost completely compliant with the reference implementation, though

Python-Markdown 3.1k Dec 30, 2022
A Python implementation of John Gruber’s Markdown with Extension support.

Python-Markdown This is a Python implementation of John Gruber's Markdown. It is almost completely compliant with the reference implementation, though

Python-Markdown 3.1k Dec 31, 2022
A Python port and library-fication of the midicsv tool by John Walker.

A Python port and library-fication of the midicsv tool by John Walker. If you need to convert MIDI files to human-readable text files and back, this is the library for you.

Tim Wedde 52 Dec 29, 2022
Mengzhan (John) code for Closed Loop Control system of Sharp Wave Ripples in Hippocampus CA3 region

ClosedLoopControl_Yu Mengzhan (John) code for Closed Loop Control system of Sharp Wave Ripples in Hippocampus CA3 region Creating Python Virtual Envir

Mengzhan (John) Liufu 1 Jan 22, 2022
A pygame implementation of John Conway's Game of Life

Game of Life A Pygame Simulation This is a Pygame implementation of the famous Conway's Game of Life. The game features a set of very simple rules: An

null 1 Jan 6, 2022
An implementation of John Conway's Game of Life.

This is an implementation of John Conway's Game of Life in Python, and a very basic and straightforward one at that.

Mae 3 Feb 11, 2022
Extendable, adaptable rewrite of django.contrib.admin

django-admin2 One of the most useful parts of django.contrib.admin is the ability to configure various views that touch and alter data. django-admin2

Jazzband 1.2k Dec 29, 2022
A rewrite of Python's builtin doctest module (with pytest plugin integration) but without all the weirdness

The xdoctest package is a re-write of Python's builtin doctest module. It replaces the old regex-based parser with a new abstract-syntax-tree based pa

Jon Crall 174 Dec 16, 2022
[rewrite 중] 코로나바이러스감염증-19(COVID-19)의 국내/국외 발생 동향 조회 API | Coronavirus Infectious Disease-19 (COVID-19) outbreak trend inquiry API

COVID-19API 코로나 바이러스 감염증-19(COVID-19, SARS-CoV-2)의 국내/외 발생 동향 조회 API Corona Virus Infectious Disease-19 (COVID-19, SARS-CoV-2) outbreak trend inquiry

Euiseo Cha 28 Oct 29, 2022
Ranger deep learning optimizer rewrite to use newest components

Ranger21 - integrating the latest deep learning components into a single optimizer Ranger deep learning optimizer rewrite to use newest components Ran

Less Wright 266 Dec 28, 2022
Extendable, adaptable rewrite of django.contrib.admin

django-admin2 One of the most useful parts of django.contrib.admin is the ability to configure various views that touch and alter data. django-admin2

Jazzband 1.2k Dec 29, 2022
Compress .dds file in ggpk to boost fps. This is a python rewrite of PoeTexureResizer.

PoeBooster Compress .dds file in ggpk to boost fps. This is a python rewrite of PoeTexureResizer. Setup Install ImageMagick-7.1.0. Download and unzip

null 3 Sep 30, 2022
SolidMusic rewrite version, need help

Telegram Streamer Bot This is rewrite version of solidmusic, but it can't be deployed now, help me to make this bot running fast and good. If anyone w

Shohih Abdul 63 Jan 6, 2022
Learning to Rewrite for Non-Autoregressive Neural Machine Translation

RewriteNAT This repo provides the code for reproducing our proposed RewriteNAT in EMNLP 2021 paper entitled "Learning to Rewrite for Non-Autoregressiv

Xinwei Geng 20 Dec 25, 2022
Rewrite ultralytics/yolov5 v6.0 opencv inference code based on numpy, no need to rely on pytorch

Rewrite ultralytics/yolov5 v6.0 opencv inference code based on numpy, no need to rely on pytorch; pre-processing and post-processing using numpy instead of pytroch.

炼丹去了 21 Dec 12, 2022
A Python3 rewrite of my original PwnedConsole project from almost a decade ago

PwnedConsoleX A CLI shell for performing queries against the HaveIBeenPwned? API to gather breach information for user-supplied email addresses. | wri

null 1 Jul 23, 2022
Genpyteal - Experiment to rewrite Python into PyTeal using RedBaron

genpyteal Converts Python to PyTeal. Your mileage will vary depending on how muc

Jason Livesay 9 Oct 19, 2022
Shellkg-py - A temporary Repository to rewrite of shellpkg in python

Shellkg-py - A temporary Repository to rewrite of shellpkg in python

null 2 Jan 26, 2022