Package managers visualization

Overview

Software Galaxies

This repository combines visualizations of major software package managers.

All visualizations are available here: http://anvaka.github.io/pm/#/

Please read operating manual - it is short and describes basic navigation principles.

Repository was create for my talk at CascadiaJS 2015

After conference update - video - slides

Friends, you are awesome! I can't express how much I appreciate all your kind words and warm feedback. It really means a world for me. Thank you!

Individual Visualizations

Each graph is indexed individually, and data is pushed to gh-pages branch of galactic-data.

Bower


indexer | demo

PHP Composer


indexer | demo

Ruby gems


indexer | demo

npm


indexer | demo

Go


indexer | demo

R language


indexer | demo

Debian


indexer | demo

Arch Linux


indexer | demo

Arch Linux + AUR


indexer | demo

NuGet


indexer | demo

Homebrew


indexer | demo

PyPI


indexer | demo

Fedora


indexer | demo

Rust Crates


indexer | demo

Elm


indexer | demo

local development

git clone https://github.com/anvaka/pm
cd pm
npm i
npm start

This will start local development sever with auto-rebuild.

Your own graphs

This section has detailed instructions about how to use the tool with your own graphs. Before you read any further, if your graph is smaller than 10k nodes, consider using ngraph.pixel or VivaGraph both should be able to provide interactive layout.

If you have an interesting graph but don't have JavaScript experience, please feel free to reach out to me and I'll try to make visualization for you (my email is [email protected]).

Otherwise, if you want to hack on your own, please keep reading.

Graph

First, you will need a graph in ngraph.graph format. The ngraph.graph has detailed documentation about how to create graph, but it also has several loaders from popular graph formats (e.g. dot, gexf)

Layout

Now that you have a graph we need to compute the layout.

If your graph is smaller than 200k nodes, consider using ngraph.offline.layout. This module was created exactly for the purpose of the pm project, it is well documented, and should be easy to get started with. You can also read layout.js of all[gems|go|bower] packages to see more examples.

If your graph is much larger than 200k nodes, then consider using ngraph.native - this module is harder to work with (as it requires C++ knowledge), but it is much faster.

The secret GitHub visualization is using ngraph.native.

Data format

Once layout is computed, we are ready to visualize. Just save the graph using ngraph.tobinary and store it along with latest positions file (produced by layout) into a folder.

The folder structure should look like this:

.
└── my-pm-data-server
    └── my-graph
        ├── manifest.json
        └── version-1
            ├── labels.json         /* this file is produced by ngraph.tobinary */
            ├── links.bin           /* this file is produced by ngraph.tobinary */
            └── positions.bin       /* this file is produced by ngraph.native   */

The file manifest.json describes what version of the graph are available and has the following content:

{
  "all": ["version-1"],
  "last": "version-1"
}

Inside my-pm-data-server we launch a web server. I personally prefer http-server. Once it is installed globally (npm i http-server -g), you can launch it like this:

http-server --cors -p 9090

This will start a local data server at http://127.0.0.1:9090/

Update the config.js in this repository to point to your data server, and your graph should be accessible at

http://127.0.0.1:8081/#/galaxy/my-graph

Note

The galactic-data follows the same data structure as described above. Use it for the reference if you need an example

The secret visualization

The last shown visualization was secret GitHub followers visualization. It shows all GitHub users who has more than two followers.

The visualization has more than 1,100,000 nodes, and renders at 60 fps when flying around. The FPS drops when you hover-over nodes to 20-30, This is because we are doing hit-testing, to find what's under cursor.

With this many nodes, it runs well in the browser. Unfortunately it requires more than 1GB of RAM. Which may or may not crash your phone browser - sorry about this.

With all warnings said, here are the links:

Feedback

Please do not hesitate to provide your feedback or bug fixes. Even if it is something small like fixing a typo - I'd be glad to hear from you!

Comments
  • Failed at start script

    Failed at start script

    When I try npm start I get the following error:

    npm ERR! Windows_NT 6.1.7601 npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "start" npm ERR! node v5.0.0 npm ERR! npm v3.3.6 npm ERR! code ELIFECYCLE npm ERR! [email protected] start: PORT=8081 node dev-server.js npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] start script 'PORT=8081 node dev-server.js'. npm ERR! This is most likely a problem with the pmviz package, npm ERR! not with npm itself. npm ERR! Tell the author that this fails on your system: npm ERR! PORT=8081 node dev-server.js npm ERR! You can get their info via: npm ERR! npm owner ls pmviz npm ERR! There is likely additional logging output above.

    Any hint?

    opened by laaposto 5
  • Graph request : Cargo's crates

    Graph request : Cargo's crates

    Rust is a growing language with a certain potential, whose build tool & package manager is Cargo. The official package registry is https://crates.io/.

    opened by AntoineSebert 4
  • Ability to find the software package in the Galaxy.

    Ability to find the software package in the Galaxy.

    It would be cool to have a means in the search to basically go on "autopilot" and have the user taken to a specific package they searched for.

    This is such a cool bit of fun. Thanks for the hard work!

    Scott

    opened by smolinari 3
  • Add NuGet &| MyGet :)

    Add NuGet &| MyGet :)

    Please add NuGet and MyGet support.

    I have found some related resources:

    • http://docs.nuget.org/Consume/Package-Visualizer.
    • https://github.com/NuGet/NuGetGallery/wiki/API-v3-Specification.

    @davidfowl, @pranavkm, is there a better (API) way to get packages and their dependencies information (preferably in JSON) for this kind of scenarios where node.js app wants to interact with NuGet like these: https://github.com/anvaka/pm/tree/master/src/galaxy/nodeDetails/templates, in order to be able to form the graphical galaxy: https://anvaka.github.io/pm/ ?

    opened by ghost 3
  • Build and deploy static React app?

    Build and deploy static React app?

    I've cloned the package and followed the instructions to visualize my graph using a local server, after determining the correct package structure.

    I want to "save" a static copy of this graph that I can host on GitHub, like the one @anvaka is hosting on his GitHub and that is in the pm/build directory. When I run npm run build in the cloned copy of the pm package, I rebuild the original React app with the package galaxies, instead of my graph.

    This may be a very novice question, but how do I use npm to generate a build of my own visualization? What should I be targeting—package structure changes, code, etc.?

    opened by brienna 2
  • sort results in search & info

    sort results in search & info

    is it possible to (by default) sort the results in the search window or the info window when you click a node? For some PMs it might make sense to sort the search results on name, or on in-degree, right now it is all random it seems. I'd prefer by default an alpabetic/numeric sort on the name of the node

    opened by job 2
  • Add more details to the

    Add more details to the "Your own graphs" section

    Hi, I followed the steps from the "Your own graphs" section

    I was able to generate the files ( positions.bin, labels.json and links.bin )

    the next step writes :

    Follow the same folder structure as in galactic-data.

    Update the config.js in this repository to point to your data server, and you are ready to explore your own graph.

    So I put the file on an express.js server into the path /public/npm/2016-06-16T00-00-00Z I added a manifest.json file into /public/npm/

    {
      "all": [
        "2016-06-16T00-00-00Z"
      ],
      "last": "2016-06-16T00-00-00Z"
    }
    

    I changed the config.js to

    export default {
      //dataUrl: '//anvaka.github.io/galactic-data/'
      dataUrl: '//localhost:8082/'
    };
    

    Then I launched anvaka/pm with npm start

    It didnt work. I don't see error in any consoles.

    I was expecting to see my own graph on http://127.0.0.1:8081/#/galaxy/npm?

    Could you provide additional information on the last step ?

    opened by HugoCornu 1
  • Add explanation of size to the descriptions

    Add explanation of size to the descriptions

    Nice and fun utility. I couldn't find an explanation of why some stars are so much larger than others. Natural tendency is to steer toward clusters and largest stars. Clusters are explained (cool!), but size is not.

    opened by rhewitt 1
  • bind 'h' and '?' to toggle the 'help' window

    bind 'h' and '?' to toggle the 'help' window

    Seems that for some scrolling the mouse is not the most intuitive way to bring up the help window, would be cool if you can add 'h' and '?' key bindings as additional methods to toggle the help window.

    opened by job 1
  • how to set color on links

    how to set color on links

    hello in my project create graph like this : graph.addLink(item.from, item.to,{color:"red"})

    then create binary file from graph like this : ToBinary.BuildBinaryGraph(graph);


    so how to get {color:"red"} in function setLinks(buffer) from project pm ?? and set my color to lineView.render(...)

    Please Help Me.

    opened by rezaokhravi 0
  • Mispositioned Graph

    Mispositioned Graph

    Hi all,

    I generated my own graph (based on this Andreis fantastic work) for BGP related routing data and in general it works. However, the graph is not positioned centrally (x=0,y=0,z=0) and therefore the link/edge colors (which base on the graphs position) are all yellow as it is shown on attached image:

    image

    Does anyone know how to fix this positioning issue?

    @anvaka : I can provide the source files if you want to debug.

    opened by AydinChavez 0
  • Visualize # of dependents?

    Visualize # of dependents?

    I'd be very interested to see a visualization that reflects either the number of direct dependents or the total dependents in a graph rooted on each package. It might use brightness and/or color. Size probably doesn't work that well because size is so influenced by distance.

    opened by estaub 0
Owner
Andrei Kashcha
I love graphs
Andrei Kashcha
A Python package that provides evaluation and visualization tools for the DexYCB dataset

DexYCB Toolkit DexYCB Toolkit is a Python package that provides evaluation and visualization tools for the DexYCB dataset. The dataset and results wer

NVIDIA Research Projects 107 Dec 26, 2022
Attractors is a package for simulation and visualization of strange attractors.

attractors Attractors is a package for simulation and visualization of strange attractors. Installation The simplest way to install the module is via

Vignesh M 45 Jul 31, 2022
Python Package for CanvasXpress JS Visualization Tools

CanvasXpress Python Library About CanvasXpress for Python CanvasXpress was developed as the core visualization component for bioinformatics and system

Dr. Todd C. Brett 5 Nov 7, 2022
Declarative statistical visualization library for Python

Altair http://altair-viz.github.io Altair is a declarative statistical visualization library for Python. With Altair, you can spend more time understa

Altair 8k Jan 5, 2023
Interactive Data Visualization in the browser, from Python

Bokeh is an interactive visualization library for modern web browsers. It provides elegant, concise construction of versatile graphics, and affords hi

Bokeh 17.1k Dec 31, 2022
Statistical data visualization using matplotlib

seaborn: statistical data visualization Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing

Michael Waskom 10.2k Dec 30, 2022
Fast data visualization and GUI tools for scientific / engineering applications

PyQtGraph A pure-Python graphics library for PyQt5/PyQt6/PySide2/PySide6 Copyright 2020 Luke Campagnola, University of North Carolina at Chapel Hill h

pyqtgraph 3.1k Jan 8, 2023
Simple, realtime visualization of neural network training performance.

pastalog Simple, realtime visualization server for training neural networks. Use with Lasagne, Keras, Tensorflow, Torch, Theano, and basically everyth

Rewon Child 416 Dec 29, 2022
Apache Superset is a Data Visualization and Data Exploration Platform

Superset A modern, enterprise-ready business intelligence web application. Why Superset? | Supported Databases | Installation and Configuration | Rele

The Apache Software Foundation 50k Jan 6, 2023
Debugging, monitoring and visualization for Python Machine Learning and Data Science

Welcome to TensorWatch TensorWatch is a debugging and visualization tool designed for data science, deep learning and reinforcement learning from Micr

Microsoft 3.3k Dec 27, 2022
Python script to generate a visualization of various sorting algorithms, image or video.

sorting_algo_visualizer Python script to generate a visualization of various sorting algorithms, image or video.

null 146 Nov 12, 2022
Statistical data visualization using matplotlib

seaborn: statistical data visualization Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing

Michael Waskom 8.1k Feb 13, 2021
Interactive Data Visualization in the browser, from Python

Bokeh is an interactive visualization library for modern web browsers. It provides elegant, concise construction of versatile graphics, and affords hi

Bokeh 14.7k Feb 13, 2021
Declarative statistical visualization library for Python

Altair http://altair-viz.github.io Altair is a declarative statistical visualization library for Python. With Altair, you can spend more time understa

Altair 6.4k Feb 13, 2021
Fast data visualization and GUI tools for scientific / engineering applications

PyQtGraph A pure-Python graphics library for PyQt5/PyQt6/PySide2/PySide6 Copyright 2020 Luke Campagnola, University of North Carolina at Chapel Hill h

pyqtgraph 2.3k Feb 13, 2021
Missing data visualization module for Python.

missingno Messy datasets? Missing values? missingno provides a small toolset of flexible and easy-to-use missing data visualizations and utilities tha

Aleksey Bilogur 3.4k Dec 29, 2022
Streaming pivot visualization via WebAssembly

Perspective is an interactive visualization component for large, real-time datasets. Originally developed for J.P. Morgan's trading business, Perspect

The Fintech Open Source Foundation (www.finos.org) 5.1k Dec 27, 2022
3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK)

PyVista Deployment Build Status Metrics Citation License Community 3D plotting and mesh analysis through a streamlined interface for the Visualization

PyVista 1.6k Jan 8, 2023
Statistical data visualization using matplotlib

seaborn: statistical data visualization Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing

Michael Waskom 8.1k Feb 18, 2021