JittorVis - Visual understanding of deep learning models

Overview

JittorVis: Visual understanding of deep learning model

Image of JittorVis

JittorVis is an open-source library for understanding the inner workings of Jittor models by visually illustrating their dataflow graphs.

Deep neural networks have achieved breakthrough performance in many tasks such as image recognition, detection, segmentation, generation, etc. However, the development of high-quality deep models typically relies on a substantial amount of trial and error, as there is still no clear understanding of when and why a deep model works. Also, the complexity of the deep neural network architecture brings difficulties to debugging and modifying the model. JittorVis facilitates the visualization of the dataflow graph of the deep neural network at different levels, which brings users a deeper understanding of the dataflow graph from the whole to the part to debug and modify the model more effectively.

JittorVis provides the visualization and tooling needed for machine learning experimentation:

  • Displaying the hierarchical structure of the model dataflow graph
  • Visualizing the dataflow graph at different levels (ops and layers)
  • Profiling Jittor programs

Features to be supported in the future:

  • Tracking and visualizing metrics such as loss and accuracy
  • Viewing line charts of weights, biases, or other tensors as they change over time
  • And much more

Related Links:

Installation

JittorVis need python version >= 3.7.

pip install jittorvis
or
pip3 install jittorvis

Usage

There are several ways to use JittorVis.

  1. Visualizing a Jittor model directly.
import jittor as jt
from jittor import Module
from jittor import nn
import numpy as np

class Model(Module):
    def __init__(self):
        self.layer1 = nn.Linear(1, 10)
        self.relu = nn.Relu() 
        self.layer2 = nn.Linear(10, 1)
    def execute (self,x) :
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        return x

model = Model()

from jittorvis import server
input = jt.float32(np.random.rand(10, 1))
server.visualize(input, model, host = '0.0.0.0')
# JittorVis start.
# server.stop()
# JittorVis stop.

Then open the link 'http://localhost:5005/static/index.html' in your browser.

  1. Visualizing an exported Jittor computational graph (an example graph can be downloaded here).
from jittorvis import server
server.run('test.pkl', host='0.0.0.0', port=5005)
# JittorVis start.
# server.stop()
# JittorVis stop.
  1. Visualizing an exported Jittor computational graph with command line interface.
jittorvis --data_path test.pkl --host='0.0.0.0' --port=5005

Visualization

JittorVis contains three main views: statistics view, navigation view, and graph structure view.

  1. Statistics view:

    The statistics view provides statistics information for the deep neural network, such as loss and accuracy.

  2. Navigation view:

    The navigation view visualizes the hierarchical structure of a Jittor model to facilitate the exploration of the model. Each tree node represents a computational node in the dataflow graph, and each leaf node represents a basic operation in the graph. Users can click one intermediate node to selected its computational nodes and turn to the graph structure view to explore their graph structure.

Drawing

  1. Graph structure view:

    The graph structure view displays the graph structure of a Jittor graph. In the graph structure view, each rectangle represents a computational node, and each link represents dataflows among computational nodes. The graph structure view has the following interactions:

    • Drag to pan
    • Scroll to zoom in and out
    • Click one computational node to explore its feature map
    • Click the top-right plus button of one computational node to explore its children
    • Click the top-right button “←” to return to the previous level of the graph
    • Right-click one computational node to explore its detailed information

Drawing

Citation

Towards Better Analysis of Deep Convolutional Neural Networks

@article {
    liu2017convolutional,
    author={Liu, Mengchen and Shi, Jiaxin and Li, Zhen and Li, Chongxuan and Zhu, Jun and Liu, Shixia},
    journal={IEEE Transactions on Visualization and Computer Graphics},
    title={Towards Better Analysis of Deep Convolutional Neural Networks},
    year={2017},
    volume={23},
    number={1},
    pages={91-100}
}

Analyzing the Training Processes of Deep Generative Models

@article {
    liu2018generative,
    author={Liu, Mengchen and Shi, Jiaxin and Cao, Kelei and Zhu, Jun and Liu, Shixia},
    journal={IEEE Transactions on Visualization and Computer Graphics},
    title={Analyzing the Training Processes of Deep Generative Models},
    year={2018},
    volume={24},
    number={1},
    pages={77-87}
}

Analyzing the Noise Robustness of Deep Neural Networks

@article {
    cao2021robustness,
    author={Cao, Kelei and Liu, Mengchen and Su, Hang and Wu, Jing and Zhu, Jun and Liu, Shixia},
    journal={IEEE Transactions on Visualization and Computer Graphics},
    title={Analyzing the Noise Robustness of Deep Neural Networks},
    year={2021},
    volume={27},
    number={7},
    pages={3289-3304}
}

The Team

JittorVis is currently maintained by the THUVIS Group. If you are also interested in JittorVis and want to improve it, Please join us!

License

JittorVis is Apache 2.0 licensed, as found in the LICENSE.txt file.

You might also like...
Cockpit is a visual and statistical debugger specifically designed for deep learning.
Cockpit is a visual and statistical debugger specifically designed for deep learning.

Cockpit: A Practical Debugging Tool for Training Deep Neural Networks

We utilize deep reinforcement learning to obtain favorable trajectories for visual-inertial system calibration.
We utilize deep reinforcement learning to obtain favorable trajectories for visual-inertial system calibration.

Unified Data Collection for Visual-Inertial Calibration via Deep Reinforcement Learning Update: The lastest code will be updated in this branch. Pleas

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021)

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021) Citation Please cite as: @inproceedings{liu2020understan

Code and data for
Code and data for "TURL: Table Understanding through Representation Learning"

TURL This Repo contains code and data for "TURL: Table Understanding through Representation Learning". Environment and Setup Data Pretraining Finetuni

This repo will contain code to reproduce and build upon understanding transfer learning

What is being transferred in transfer learning? This repo contains the code for the following paper: Behnam Neyshabur*, Hanie Sedghi*, Chiyuan Zhang*.

PyTorch implementation of DirectCLR from paper Understanding Dimensional Collapse in Contrastive Self-supervised Learning
PyTorch implementation of DirectCLR from paper Understanding Dimensional Collapse in Contrastive Self-supervised Learning

DirectCLR DirectCLR is a simple contrastive learning model for visual representation learning. It does not require a trainable projector as SimCLR. It

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning Authors: Tao Yu* Yichi Zhang* Zhiru Zhang Christopher De Sa *: Equal Contri

Ever felt tired after preprocessing the dataset, and not wanting to write any code further to train your model? Ever encountered a situation where you wanted to record the hyperparameters of the trained model and able to retrieve it afterward? Models Playground is here to help you do that. Models playground allows you to train your models right from the browser. PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+
PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

PaddlePaddle Vision Transformers State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 🤖 PaddlePaddle Visual Transformers (PaddleViT or

Comments
  • Add more easy-use function interface

    Add more easy-use function interface

    Currently, visualize a computation graph needs to export pkl file and load in jittorvis, it is not straightforward, we can do it in a better way:

    import jittorvis
    import jittor as jt
    from jittor.models import resnet18
    
    model = resnet18()
    img = jt.rand((1,3,100,100))
    
    jittorvis.visualize_model(model, img, ip=xxxx, port=xxxx, ...)
    
    opened by Jittor 2
Owner
thu-vis
Tsinghua Visual Analytics Group
thu-vis
improvement of CLIP features over the traditional resnet features on the visual question answering, image captioning, navigation and visual entailment tasks.

CLIP-ViL In our paper "How Much Can CLIP Benefit Vision-and-Language Tasks?", we show the improvement of CLIP features over the traditional resnet fea

null 310 Dec 28, 2022
Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for

Abdultawwab Safarji 7 Nov 27, 2022
[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

Towards Understanding and Mitigating Social Biases in Language Models This repo contains code and data for evaluating and mitigating bias from generat

Paul Liang 42 Jan 3, 2023
PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.

PySlowFast PySlowFast is an open source video understanding codebase from FAIR that provides state-of-the-art video classification models with efficie

Meta Research 5.3k Jan 3, 2023
The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.

SuperGen The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding. Requirements Before running, you

Yu Meng 38 Dec 12, 2022
Code for the CVPR 2021 paper: Understanding Failures of Deep Networks via Robust Feature Extraction

Welcome to Barlow Barlow is a tool for identifying the failure modes for a given neural network. To achieve this, Barlow first creates a group of imag

Sahil Singla 33 Dec 5, 2022
CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

CLIP (Contrastive Language–Image Pre-training) Experiments (Evaluation) Model Dataset Acc (%) ViT-B/32 (Paper) CIFAR100 65.1 ViT-B/32 (Our) CIFAR100 6

Myeongjun Kim 52 Jan 7, 2023
Visual Adversarial Imitation Learning using Variational Models (VMAIL)

Visual Adversarial Imitation Learning using Variational Models (VMAIL) This is the official implementation of the NeurIPS 2021 paper. Project website

null 14 Nov 18, 2022
Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

Machine Learning From Scratch About Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The purpose

Erik Linder-Norén 21.8k Jan 9, 2023