Welcome to graph-app-kit
Turn your graph data into a secure and interactive visual graph app in 15 minutes!
Why
This open source effort puts together patterns the Graphistry team has reused across many graph projects as teams go from code-heavy Jupyter notebook experiments to deploying streamlined analyst tools. Whether building your first graph app, trying an idea, or wanting to check a reference, this project aims to simplify that process. It covers pieces like: Easy code editing and deployment, a project stucture ready for teams, built-in authentication, no need for custom JS/CSS at the start, batteries-included data + library dependencies, and fast loading & visualization of large graphs.
What
-
Minimal core: The barebones dashboard server. In provides a StreamLit docker-compose container with PyData ecosystem libraries and examples of visualizing data from various systems. Install it, plug in credentials to various web services like cloud databases and a free Graphistry Hub visualization account, and launch.
-
Full core: Initially for AWS, the full core bundles adds to the docker-compose system: Accounts, Jupyter notebooks for authoring, serves StreamLit dashboards with both public + private zones, and runs Graphistry/RAPIDS locally on the same server. Launch with on click via the Cloud Formation template.
-
Full core + DB: DB-specific variants are the same as minimal/full, and add simpler DB-specific quick launching/connecting.
Get started
Quick (Local code) - minimal core + third-party connectors
# Minimal core
git clone https://github.com/graphistry/graph-app-kit.git
cd graph-app-kit/src/docker
sudo docker-compose build
# Optional: Edit src/docker/.env (API accounts), docker-compose.yml: Auth, ports, ...
# Launch
sudo docker-compose up -d
sudo docker-compose logs -f -t --tail=100
=> http://localhost:8501/
To add views and relaunch:
# Add dashboards @ src/python/views/<your_custom_view>/__init__.py
sudo docker-compose up -d --force-recreate
Quick Launchers - minimal/full core
- Quick launch options:
- Public + protected Streamlit dashboards, Jupyter notebooks + editing, Graphistry, RAPIDS
- Login to web UI as
admin
/i-instanceid
-> file uploader, notebooks, ... - Dashboards:
/public/dash
and/private/dash
- More info
Admin:
# launch logs
tail -f /var/log/cloud-init-output.log -n 1000
# app logs
sudo docker ps
sudo docker logs -f -t --tail=1 MY_CONTAINER
# restart a graphistry container
cd graphistry && sudo docker-compose restart MY_CONTAINER
# restart caddy (Caddy 1 override)
cd graphistry && sudo docker-compose -f docker-compose.gak.graphistry.yml up -d caddy
# run streamlit
cd graph-app-kit/public/graph-app-kit && docker-compose -p pub run -d --name streamlit-pub streamlit
cd graph-app-kit/private/graph-app-kit && docker-compose -p priv run -d --name streamlit-priv streamlit
Minimal: Open Streamlit, ssh to connect/add free Graphistry Hub username/pass:
Database-specific: Amazon Neptune, TigerGraph
-
Main configurations and extensions: Database connectors, authentication, notebook-based editing, and more
The pieces
Core
- Prebuilt Python project structure ready for prototyping
- Streamlit quick self-serve dashboarding
- Graphistry point-and-click GPU-accelerated visual graph analytics
- Data frames: Data wrangling via Pandas, Apache Arrow, RAPIDS (ex: cuDF), including handling formats such as CSV, XLS, JSON, Parquet, and more
- Standard Docker and docker-compose cross-platform deployment
GPU acceleration (optional)
If GPUs are present, graph-app-kit
leverages GPU cloud acceleration:
-
GPU Analytics: RAPIDS and CUDA already setup for use if run with an Nvidia docker runtime - cudf GPU dataframes, BlazingSQL GPU SQL, cuGraph GPU graph algorithms, cuML libraries, and more
-
GPU Visualization: Connect to an external Graphistry server or, faster, run on the same GPU server
Prebuilt integrations & recipes
graph-app-kit
works well with the Python data ecosystem (pandas, cudf, PySpark, SQL, ...) and we're growing the set of builtins and recipes:
-
Graph databases
-
Collaborations welcome!
-
Jupyter notebooks: Use quick launchers or integrations guide for web-based live editing of dashboards by sharing volume mounts between Jupyter and Streamlit
-
Caddy: Reverse proxy for custom URLs, automatic LetsEncrypt TLS certificates, multiple sites on the same domain, pluggable authentication (see integrations guide)
Contribute
We welcome all sorts of help!
- Deployment: Docker, cloud runners, ...
- Dependencies: Common graph packages
- Connectors: Examples for common databases and how to get a lot of data out
- Demos!
See develop.md for more contributor information