Portal is the fastest way to load and visualize your deep neural networks on images and videos ๐Ÿ”ฎ

Overview

Portal

Build Tests Join Datature Slack

Portal is the fastest way to load and visualize your deep learning models. We are all sick of wrangling a bunch of cv2 or matplotlib codes to test your models - especially on videos. We created portal to help teams, engineers, and product managers interactively test their model on images and videos, inference thresholds, IoU values and much more.

Portal is an open-source browser-based app written in TypeScript, React and Flask.

Made with โ™ฅ by Datature


Portal User Experience

Portal works on both images and videos - bounding boxes and masks - allowing you to use it as a sandbox for testing your model's performance. Additionally, Portal supports Datature Hub, TensorFlow and DarkNet models (PyTorch Support Incoming) and runs on either our electron app or your browser.

Portal User Experience Portal User Experience

Setting up Portal

Portal can be used as a Web Application or through downloading and installing our Electron package via Portal Releases.

Running Portal as a Web Application

Portal is built using python 3.7. Ensure that you have this version (and up) before beginning. Clone the repository and then navigate to the directory where requirements.txt is and install all necessary dependencies and setup using setup.sh:

git clone https://github.com/datature/portal
cd portal
pip3 install -r requirements.txt
./setup.sh

Running the following command will open the Portal application on the browser via http://localhost:9449.

If you wish to run the application on gpu, add a trailing --gpu flag (This only works for TensorFlow Models)

python3 portal.py

Using Virtual Environment

If you'd like to use virtual environments for this project - you can use a helpful script below to before activating the virtualenv -

./setup-virtualenv.sh

Running from Portal Executable

Portal comes with an installable version that runs on electron.js - this helps to provide a desktop application feel and ease of access of setting up. To install, please download the latest Portal Releases and run the Portal installer for your OS.

Navigating Portal

On starting Portal or navigating to http://localhost:9449 - The following steps details how you can load your YOLO or TensorFlow model on your image folders. To begin, let's assume we want to register a tf2.0 model. On Portal, a concept we use is that you can register multiple models but load one at each time.

Registering and Loading Portal

Start by clicking on the + sign and adding the relevant filepaths, e.g. /user/portal/downloads/MobileNet/ and a name. You will be prompted to load the model as seen below. Simply click on the model you'd like to load and the engine wil

Register Model Load Model

Loading Your Images / Videos

To load your dataset (images / videos), click on the Open Folders button in the menu and paste your folder path to your dataswr. Once you are done, press the enter button. The images should appear in the asset menu below. You can load and synchronize multiple folders at once on Portal.

Load Assets

Running Inferences

Click on any image or video, press Analyze, and Portal will make the inference and render the results. You can then adjust the confidence threshold or filter various classes as needed. Note that Portal run inferences on videos frame-by-frame, so that will take some time. You can change the inference settings, such as IoU or Frame Settings under Advanced Settings.

Image Prediction Image Prediction

Portal works on both Mask and Bounding Box models. For detailed documentations about advanced features of Portal can be found here : Portal Documentation

Sample Weights

We have provided sample weights for you to test portal:

Dataset Description Download Link
YOLO-v3 DarkNet Model based off pjreddie/darknet YOLOv3
SSD MobileNet V2 FPNLite 640x640 Tensorflow Model from tensorflow/models MobileNet
Comments
  • Show prediction labels

    Show prediction labels

    Hey there, amazing tool, thanks for sharing!

    I wonder if it is possible to draw the class label along with the bounding box for each prediction (possibly with the possibility of turning on/off the feature)

    Actually, this bit of the code is supposed to do that, is that the case this works only for tf2 models? https://github.com/datature/portal/blob/a972880183c3465f1851a6b360a87c47691495c5/src/engine/server/utilities/prediction_utilities.py#L616-L631

    feature request 
    opened by alemelis 5
  • How to install Datuture Hub

    How to install Datuture Hub

    After running requirements.txt:

    ERROR: Could not find a version that satisfies the requirement datature-hub==0.2.0 (from versions: none) ERROR: No matching distribution found for datature-hub==0.2.0

    When I tried installing datature-hub via pip install datature-hub https://pypi.org/project/datature-hub/ error =

    ERROR: Could not find a version that satisfies the requirement datature-hub (from versions: none)
    ERROR: No matching distribution found for datature-hub
    
    opened by getcontrol 2
  • Bulk Analysis Feature

    Bulk Analysis Feature

    HotKey B and the menu item bulk analysis button will invoke bulkAnalysis() method which goes through the entire assetList to get inference for each of the asset.

    Under the Advance Settings, the User is able to choose if they want the bulk analysis to include Image Only, Video Only or Image and Video

    image

    Includes end bulk analysis and end video prediction. Does not work for ending image prediction

    opened by beesaycheese 2
  • Call the backend for a list of cached predictions and bookmarked the thumbnails

    Call the backend for a list of cached predictions and bookmarked the thumbnails

    Overall changes made:

    • Thumbnail of the pictures that has been cached has a bookmark icon wrapped with a ToolTip info "Inference is Cached by Model"
    • Each time a picture with cached predictions is selected, it will silently call the prediction route. (Remove the progress bar but the spinner on the annotation menu still remains)

    How the changes are made:

    1. Add a isCached field in the AssetAPIObject type --> which will be used to check if the asset needs a bookmark icon on its thumbnail
    2. Add a cachedList array which will be called and updated each time the imagebar needs to be updated (aka, refer below)
    • ComponentDidMount
    • ComponentDidUpdate (when the global loadedModel is updated)
    • getInference (when a prediction is being called)
    opened by beesaycheese 2
  • Hotfix/cleanup executable builds

    Hotfix/cleanup executable builds

    Change the name from annotator to portal Bash jobs all exits when an error occurs For the setup bash files, added a condition that checks for the necessary files that needs to be transferred to portal-build first Pyinstaller error for macos change python3 portal-build/portal.py to python3 portal.py

    opened by beesaycheese 1
  • Feature/single folder build

    Feature/single folder build

    Mac:

    git clone https://github.com/datature/portal
    chmod u+x ./setup-virtualenv.sh && ./setup-virtualenv.sh
    . portal-build/.venv/bin/activate
    python3 portal-build/portal.py (additional flags include --electron and --gpu)
    

    Win:

    git clone https://github.com/datature/portal
    setup-virtualenv.sh
    portal-build\.venv\Scripts\activate
    python portal-build\portal.py  (additional flags include --electron and --gpu)
    

    Additional Stuff changed:

    1. Toaster logic (On browser mode, the blue toaster will stay forever until it detects the backend)
    2. Set a global isConnected state
    3. Disable inference buttons when there is no loaded model. On hover, a tooltip info will appear
    4. OnComponentDidMount, the model component will call the get model list api until it gets a postive ans. During the wait, the model button is in skeleton mode
    5. GPU feature to request for a restart
    opened by beesaycheese 1
  • Features for V0.55

    Features for V0.55

    1. Changes to the Cache saving and loading methods
    • Instead of saving the entire model class object, the model class is deconstructed on save, and reconstructed on load
    1. Changes to loaded model location in the code
    • Loaded models are now fully integrated into their respective model class
    • Docstrings for this change has been updated.
    1. Added new autosave toggling feature
    • Introduced new variable var/cache.var.
    • Enabling/Disabling autosave will update the var/cache.var flag.
    • Whenever Portal starts, autosave will be set based on the value in var/cache.var.
    • Autosave is defaulted to false on first start.
    • Enabling autosave will immediately create the portalCache file.
    • Disabling autosave will immediately delete the portalCache file.
    1. Created support for Hosted Deployment into Portal
    • Added new tab in model registration interface called "Datature API (Coming Soon)"
    • (The tab is currently greyed out)
    • Upon clicking the tab users can enter Name, API Link, and Project Secret to register their API as a model.
    • Subsequent Steps to load/make predictions will not be different from previous methods.
    1. Introduced option for certain model types to be saved in cache.
    • With the introduction of Datature API, saving the credentials (API link and project secret) could be a security breach.
    • This new option allows newly created custom model types to specify if they want their credentials to be saved in cache.
    • All registered models that have the save_in_cache flag turned off will not have their credentials saved in cache.
    • Image predictions tagged to this model will still be saved in cache nonetheless (The registered model key used for indexing has already been hashed).
    1. Changed default video frame sampling rate from 20 to 1.
    2. Moved and renamed use_gpu from cache/use_gpu to var/gpu.var.
    3. hub_models, var and cache folders will no longer be persistent (removed from github), and will instead be dynamically created upon first startup.
    opened by marcus-neo 0
  • :sparkles: [DRAFT] feat: add graphing visualization portal objects

    :sparkles: [DRAFT] feat: add graphing visualization portal objects

    Changes

    • add new package recharts
    • add new button for visualizing the number of recognized objects over time
    • add feature for video seeking on the graph

    Success Criteria

    • [x] Render timeline graph
    • [x] Timeline graph shows composition of objects (each frame is one datapoint)
    • [x] Selecting the graph shows you the number of particular objects on a given frame
    • [x] On selection of datapoint of a video, the video will seek to the timepoint

    Screenshots

    New Buttton Bar Chart for Images Line chart for Videos
    opened by ShenyiCui 0
  • [BUG] Portal runtime is unresponsive.

    [BUG] Portal runtime is unresponsive.

    Describe the bug Portal runtime is unresponsive when launching the application.

    To Reproduce Running this build: https://github.com/datature/portal/releases/tag/v0.5.5 On MacOs

    Uncaught (in promise) Error: Network Error
        at e.exports (835-91f7fb22de8eba0dd168.js:1)
        at XMLHttpRequest.p.onerror (835-91f7fb22de8eba0dd168.js:1)
    localhost:9449/api/project/assets:1 
    Failed to load resource: net::ERR_CONNECTION_REFUSED
    835-91f7fb22de8eba0dd168.js:1 
    Uncaught (in promise) Error: Network Error
        at e.exports (835-91f7fb22de8eba0dd168.js:1)
        at XMLHttpRequest.p.onerror (835-91f7fb22de8eba0dd168.js:1)```
    
    bug 
    opened by AlexVialaBellander 1
  • [Questions] - TensorFlow.js Support?

    [Questions] - TensorFlow.js Support?

    Ask your question and elaborate its context I see you mentioning TensorFlow models, but does Portal support TensorFlow.js? If so that would be huge for me.

    Additional context If you support TensorFlow.js I'd love to do a blog post and video demo to put on Twitter!

    question 
    opened by GantMan 1
  • Unable to load model file

    Unable to load model file

    Hello everyone, thanks a lot for developing this application. Looks super useful and would make all our lives much easier. I have installed your portal application for windows provided in the link (Portal-0.5.0-windows.exe). After installing, I am facing two errors:

    1. I am trying to load yolov3. I have downloaded the model using the link given in the readme. Now, I am trying to load the model in the + but I get the error stating Failed to register model. The model file directory is as follows:
    โ””โ”€โ”€ YOLOv3 ImageNet
        โ”œโ”€โ”€  __MACOSX
        โ””โ”€โ”€ YOLOv3 ImageNet
              โ”œโ”€ coco (names file)
              โ”œโ”€ yolov3 (CFG file)
              โ””โ”€ yolov3 (WEIGHTS file)
    
    1. I am also continuously getting a prompt error saying Portal runtime is unresponsive. (Also can be seen in the snap below)

    image

    bug question 
    opened by rishabh700 2
Releases(v0.5.5)
  • v0.5.5(Aug 13, 2021)

    Portal First Update

    • Added Cache Folder and System Stability
    • Suspect that there might be issues in Windows build. Will add stable fix for executable in future.
    • For issues and questions on installation, post a question either on Datature Community or open an issue

    Key Features

    - Loading Interface for Datature Hub and Datature API (Coming Soon)
    - Added Image Labels Toggle to Hotkey L
    - Improvement of Cache System
    - Added Early Work on Datature API Sync
    
    Source code(tar.gz)
    Source code(zip)
    Portal-0.5.5-macos.dmg(306.61 MB)
    Portal-0.5.5-windows.exe(456.99 MB)
  • v0.5.0(Jul 27, 2021)

    Portal First Release

    • Executable runs on Python 3.7 and uses an Electron.JS wrapper to wrap around the Portal Engine and WebApp
    • Portal-0.5.0-macos.dmg and Portal-0.5.0-windows.exe are built by Github Actions, SHA256 hashes will be provided in subsequent updates
    • Users can build this exact build using ./build-electron.sh
    • For issues and questions on installation, post a question either on Datature Community or open an issue

    Key Features

    - Loading of TensorFlow 2.0 and DarkNet YOLO Models
    - Inferences on Video and Images
    - Updating of Thresholds, IoU and Class Filters
    - Improved Folder Targeting and Syncing
    - Included Datature Hub Loader
    - Added Progress Bar and ETA for Video Predictions (Which Usually Takes ~5-10 Minutes)
    
    Source code(tar.gz)
    Source code(zip)
    Portal-0.5.0-macos.dmg(305.82 MB)
    Portal-0.5.0-windows.exe(456.57 MB)
Owner
Datature
Powering breakthrough AI projects - With love. Without Code.
Datature
Visual Computing Group (Ulm University) 99 Nov 30, 2022
Visualization toolkit for neural networks in PyTorch! Demo -->

FlashTorch A Python visualization toolkit, built with PyTorch, for neural networks in PyTorch. Neural networks are often described as "black box". The

Misa Ogura 692 Dec 29, 2022
Making decision trees competitive with neural networks on CIFAR10, CIFAR100, TinyImagenet200, Imagenet

Neural-Backed Decision Trees ยท Site ยท Paper ยท Blog ยท Video Alvin Wan, *Lisa Dunlap, *Daniel Ho, Jihan Yin, Scott Lee, Henry Jin, Suzanne Petryk, Sarah

Alvin Wan 556 Dec 20, 2022
GNNLens2 is an interactive visualization tool for graph neural networks (GNN).

GNNLens2 is an interactive visualization tool for graph neural networks (GNN).

Distributed (Deep) Machine Learning Community 143 Jan 7, 2023
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX (.onnx, .pb, .pbtxt), Keras (.h5, .keras), Tens

Lutz Roeder 20.9k Dec 28, 2022
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX, TensorFlow Lite, Keras, Caffe, Darknet, ncnn,

Lutz Roeder 16.3k Sep 27, 2021
null 131 Jun 25, 2021
Visualization Toolbox for Long Short Term Memory networks (LSTMs)

Visualization Toolbox for Long Short Term Memory networks (LSTMs)

Hendrik Strobelt 1.1k Jan 4, 2023
A collection of infrastructure and tools for research in neural network interpretability.

Lucid Lucid is a collection of infrastructure and tools for research in neural network interpretability. We're not currently supporting tensorflow 2!

null 4.5k Jan 7, 2023
Neural network visualization toolkit for tf.keras

Neural network visualization toolkit for tf.keras

Yasuhiro Kubota 262 Dec 19, 2022
Code for visualizing the loss landscape of neural nets

Visualizing the Loss Landscape of Neural Nets This repository contains the PyTorch code for the paper Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer

Tom Goldstein 2.2k Dec 30, 2022
Pytorch implementation of convolutional neural network visualization techniques

Convolutional Neural Network Visualizations This repository contains a number of convolutional neural network visualization techniques implemented in

Utku Ozbulak 7k Jan 3, 2023
Using / reproducing ACD from the paper "Hierarchical interpretations for neural network predictions" ๐Ÿง  (ICLR 2019)

Hierarchical neural-net interpretations (ACD) ?? Produces hierarchical interpretations for a single prediction made by a pytorch neural network. Offic

Chandan Singh 111 Jan 3, 2023
pytorch implementation of "Distilling a Neural Network Into a Soft Decision Tree"

Soft-Decision-Tree Soft-Decision-Tree is the pytorch implementation of Distilling a Neural Network Into a Soft Decision Tree, paper recently published

Kim Heecheol 262 Dec 4, 2022
A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

Souvik Pratiher 16 Nov 17, 2021
Convolutional neural network visualization techniques implemented in PyTorch.

This repository contains a number of convolutional neural network visualization techniques implemented in PyTorch.

null 1 Nov 6, 2021
JittorVis - Visual understanding of deep learning model.

JittorVis is a deep neural network computational graph visualization library based on Jittor.

null 182 Jan 6, 2023
Interpretability and explainability of data and machine learning models

AI Explainability 360 (v0.2.1) The AI Explainability 360 toolkit is an open-source library that supports interpretability and explainability of datase

null 1.2k Dec 29, 2022
Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Class Activation Map methods implemented in Pytorch pip install grad-cam โญ Comprehensive collection of Pixel Attribution methods for Computer Vision.

Jacob Gildenblat 6.5k Jan 1, 2023