A toolbox to iNNvestigate neural networks' predictions!

Overview

iNNvestigate neural networks! Tweet

GitHub package version Keras package version License: BSD-2 Build Status

Different explanation methods on ImageNet.

Table of contents

Introduction

In the recent years neural networks furthered the state of the art in many domains like, e.g., object detection and speech recognition. Despite the success neural networks are typically still treated as black boxes. Their internal workings are not fully understood and the basis for their predictions is unclear. In the attempt to understand neural networks better several methods were proposed, e.g., Saliency, Deconvnet, GuidedBackprop, SmoothGrad, IntergratedGradients, LRP, PatternNet&-Attribution. Due to the lack of a reference implementations comparing them is a major effort. This library addresses this by providing a common interface and out-of-the-box implementation for many analysis methods. Our goal is to make analyzing neural networks' predictions easy!

If you use this code please star the repository and cite the following paper:

Alber, M., Lapuschkin, S., Seegerer, P., Hägele, M., Schütt, K. T., Montavon, G., Samek, W., Müller, K. R., Dähne, S., & Kindermans, P. J. (2019). iNNvestigate neural networks! Journal of Machine Learning Research, 20.

@article{JMLR:v20:18-540,
author  = {Maximilian Alber and Sebastian Lapuschkin and Philipp Seegerer and Miriam H{{\"a}}gele and Kristof T. Sch{{\"u}}tt and Gr{{\'e}}goire Montavon and Wojciech Samek and Klaus-Robert M{{\"u}}ller and Sven D{{\"a}}hne and Pieter-Jan Kindermans},
title   = {iNNvestigate Neural Networks!},
journal = {Journal of Machine Learning Research},
year    = {2019},
volume  = {20},
number  = {93},
pages   = {1-8},
url     = {http://jmlr.org/papers/v20/18-540.html}
}

Installation

iNNvestigate can be installed with the following commands. The library is based on Keras and therefore requires a supported Keras-backend o(Currently only the TensorFlow backend is supported. We test with Python 3.6, TensorFlow 1.12 and Cuda 9.x.):

pip install innvestigate
# Installing Keras backend
pip install [tensorflow | theano | cntk]

To use the example scripts and notebooks one additionally needs to install the package matplotlib:

pip install matplotlib

The library's tests can be executed via:

git clone https://github.com/albermax/innvestigate.git
cd innvestigate
python setup.py test

Usage and Examples

The iNNvestigate library contains implementations for the following methods:

  • function:
    • gradient: The gradient of the output neuron with respect to the input.
    • smoothgrad: SmoothGrad averages the gradient over number of inputs with added noise.
  • signal:
    • deconvnet: DeConvNet applies a ReLU in the gradient computation instead of the gradient of a ReLU.
    • guided: Guided BackProp applies a ReLU in the gradient computation additionally to the gradient of a ReLU.
    • pattern.net: PatternNet estimates the input signal of the output neuron.
  • attribution:
    • input_t_gradient: Input * Gradient
    • deep_taylor[.bounded]: DeepTaylor computes for each neuron a root point, that is close to the input, but which's output value is 0, and uses this difference to estimate the attribution of each neuron recursively.
    • pattern.attribution: PatternAttribution applies Deep Taylor by searching root points along the signal direction of each neuron.
    • lrp.*: LRP attributes recursively to each neuron's input relevance proportional to its contribution of the neuron output.
    • integrated_gradients: IntegratedGradients integrates the gradient along a path from the input to a reference.
  • miscellaneous:
    • input: Returns the input.
    • random: Returns random Gaussian noise.

The intention behind iNNvestigate is to make it easy to use analysis methods, but it is not to explain the underlying concepts and assumptions. Please, read the according publication(s) when using a certain method and when publishing please cite the according paper(s) (as well as the iNNvestigate paper). Thank you!

All the available methods have in common that they try to analyze the output of a specific neuron with respect to input to the neural network. Typically one analyses the neuron with the largest activation in the output layer. For example, given a Keras model, one can create a 'gradient' analyzer:

import innvestigate

model = create_keras_model()

analyzer = innvestigate.create_analyzer("gradient", model)

and analyze the influence of the neural network's input on the output neuron by:

analysis = analyzer.analyze(inputs)

To analyze a neuron with the index i, one can use the following scheme:

analyzer = innvestigate.create_analyzer("gradient",
                                        model,
					neuron_selection_mode="index")
analysis = analyzer.analyze(inputs, i)

Let's look at an example (code) with VGG16 and this image:

Input image.

import innvestigate
import innvestigate.utils
import keras.applications.vgg16 as vgg16

# Get model
model, preprocess = vgg16.VGG16(), vgg16.preprocess_input
# Strip softmax layer
model = innvestigate.utils.model_wo_softmax(model)

# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model)

# Add batch axis and preprocess
x = preprocess(image[None])
# Apply analyzer w.r.t. maximum activated output-neuron
a = analyzer.analyze(x)

# Aggregate along color channels and normalize to [-1, 1]
a = a.sum(axis=np.argmax(np.asarray(a.shape) == 3))
a /= np.max(np.abs(a))
# Plot
plt.imshow(a[0], cmap="seismic", clim=(-1, 1))

Analysis image.

Trainable methods

Some methods like PatternNet and PatternAttribution are data-specific and need to be trained. Given a data set with train and test data, this can be done in the following way:

import innvestigate

analyzer = innvestigate.create_analyzer("pattern.net", model)
analyzer.fit(X_train)
analysis = analyzer.analyze(X_test)

Tutorials

In the directory examples one can find different examples as Python scripts and as Jupyter notebooks:


To use the ImageNet examples please download the example images first (script).

More documentation

... can be found here:

  • Alber, M., Lapuschkin, S., Seegerer, P., Hägele, M., Schütt, K. T., Montavon, G., Samek, W., Müller, K. R., Dähne, S., & Kindermans, P. J. (2019). INNvestigate neural networks! Journal of Machine Learning Research, 20.](https://jmlr.org/papers/v20/18-540.html)
    @article{JMLR:v20:18-540,
    author  = {Maximilian Alber and Sebastian Lapuschkin and Philipp Seegerer and Miriam H{{\"a}}gele and Kristof T. Sch{{\"u}}tt and Gr{{\'e}}goire Montavon and Wojciech Samek and Klaus-Robert M{{\"u}}ller and Sven D{{\"a}}hne and Pieter-Jan Kindermans},
    title   = {iNNvestigate Neural Networks!},
    journal = {Journal of Machine Learning Research},
    year    = {2019},
    volume  = {20},
    number  = {93},
    pages   = {1-8},
    url     = {http://jmlr.org/papers/v20/18-540.html}
    }
    
  • https://innvestigate.readthedocs.io/en/latest/

Contributing

If you would like to contribute or add your analysis method please open an issue or submit a pull request.

Releases

Can be found here.

Comments
  • LRP for semantic segmentation

    LRP for semantic segmentation

    Hi,

    Thank you very much for this nice library. I think it is a great initiative to have a collection of neural network analysis tools!

    I tried to use iNNvestigate to get some insights into semantic segmentation neural networks. I created a toy-task to segment the mnist images using a small U-net architecture. The desired target images were created by thresholding the input images at 0.5. I encountered the following problems when running different variants of LRP analysis:

    • Running the same analysis for the same output neuron for the same input multiple times results in different relevance maps (e.g., the relevance at the same position can be sometimes negative and sometime positive).
    • The relevance map doesn't sum up to the output value of the analysed neuron.

    Please find attached a jupyter notebook (as a txt file, since github doesn't support the ipynb extension) and a generated pdf with my code, that I used for this toy-task. I would appreciate your feedback and any hints on how to use LRP to analyse semantic segmentation models.

    Best, Grzegorz mnist_unet.pdf mnist_unet.txt

    opened by gchlebus 20
  • Notebooks

    Notebooks

    We use (ipy)notebooks as main way to introduce people to the api. "Learning by doing".

    The following notebooks and contents should be part of the first release:

    • A) [core reversal code:] walk through the idea of inverting the graph, show how to implement the gradient by inverting each layer, then a slightly more advanced example by showing how to create deconvnet and guided backprop.

      • best done with inception v3 => fast network and imagenet pics look great.
      • If there are some nice visualization/sketches on how the reversion works, we can use them also in the paper. Parts of the text can also be shared.
    • B) [mnist complete workflow:] the same as the mnist example right now, just with more comments on how things work. And the code should be structured more nicely, i.e, not using those function, more in a step by step fashion.

      • the pics at the moment look horrible, I think this is due to the training. Best look into the cat-paper and try to reproduce the results, in case ask Sebastian or Pieter-Jan how the got them.
    • C) [imagenet application] a step-by-step example with imagenet, again similar to the all_methods.py. The scope here is to reuse the patterns and let the people know how the can use our applications submodule.

    • D) [imagenet network comparison] Same as the C only the outcome should be a square with n networks as rows and n methods as columns. The code can and will be a bit more messy as different networks need different preprocessing functions and image sizes. Overall it should be doable as the key information is already present in the dictionary returned by innvestigate.applications.

      • Reference C and delimit the scope to show the power of innvestigate and on how to compare different networks.
      • The plots of this example will be used in the final paper.
    • E) [imagent pattern training] similar to the current train_***.py. Focus on how to train patterns for a large network. In the best case this example shows how to train with several gpus (means setting the parameter gpus=X).

    • D) [perturbation mnist] an example notebook on how to use perturbation analysis with mnist that everybody can run.

    • E) [perturbation imagenet] an example notebook on how to use perturbation analysis with imagenet.

    • F) [LRP/DT intro] Sebastian might wants to add an LRP/DT notebook.

    I think if there exists first a working python-script it is easy to create the notebook. The advantage would be one can run the examples easily via ssh. The drawback is the code is doubled and when changed one needs to change two places.

    enhancement ForFirstRelease documentation 
    opened by albermax 14
  • Setup script.

    Setup script.

    Add needed packages to setup script. Test by using empty venv and install package.

    Known needed modules: pillow, numpy, scipy, keras.

    Note: it should be up to the user which backend for keras he installs.

    bug ForFirstRelease code 
    opened by albermax 14
  • NoneType object has no attribute '_keras_shape'

    NoneType object has no attribute '_keras_shape'

    The innvestigate analyzer is not able to analyze the following network (created with keras). I suppose that it is related to the Embedding Layer from keras.

    from keras import Sequential
    from keras.layers import Dense, Conv1D, Embedding, GlobalMaxPooling1D
    import numpy as np
    import innvestigate
    
    model = Sequential()
    model.add(Embedding(input_dim=219, output_dim=8))
    model.add(Conv1D(filters=64, kernel_size=8, padding='valid', activation='relu'))
    model.add(GlobalMaxPooling1D())
    model.add(Dense(16, activation='relu'))
    model.add(Dense(2, activation=None))
    
    #test
    model.predict(np.random.randint(1,219, (1,100)))  # [[0.04913538 0.04234646]]
    
    analyzer = innvestigate.create_analyzer('lrp.epsilon', model, neuron_selection_mode=max_activation, **{'epsilon': 1})
    analyzer.analyze(np.random.randint(1, 219, (1,100)))
    

    I get the following error when trying to execute the above:

    Traceback (most recent call last):
      File "test.py", line 357, in <module>
        analyzer.analyze(np.random.randint(1,218, (1,100)))
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 469, in analyze
        self.create_analyzer_model()
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 407, in create_analyzer_model
        model, stop_analysis_at_tensors=stop_analysis_at_tensors)
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/relevance_based/relevance_analyzer.py", line 457, in _create_analysis
        return super(LRP, self)._create_analysis(*args, **kwargs)
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 696, in _create_analysis
        return_all_reversed_tensors=return_all_reversed_tensors)
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 888, in reverse_model
        "stop_mapping_at_tensors": local_stop_mapping_at_tensors,
      File "/home/alex/virtualEnvs/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/relevance_based/relevance_analyzer.py", line 481, in _default_reverse_mapping
        Xs, Ys, reversed_Ys, reverse_state)
      File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 589, in _gradient_reverse_mapping
        return ilayers.GradientWRT(len(Xs), mask=mask)(Xs+Ys+reversed_Ys)
      File "/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py", line 497, in __call__
        arguments=user_kwargs)
      File "/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py", line 565, in _add_inbound_node
        output_tensors[i]._keras_shape = output_shapes[i]
    AttributeError: 'NoneType' object has no attribute '_keras_shape'
    

    I tested with the newest version 1.0.4 of the innvestigate package.

    enhancement 
    opened by alewarne 12
  • Multi-View CNN support

    Multi-View CNN support

    I want to use this terrific tool on an MVCNN type of network, which require to feed the keras model a list of ndarrays rather than a single ndarray, I have tried the naive approach and fed the analyzer a list of ndarrays but it cant get past the collection of standardize_user_data methods. Is there a way to use innvestigate on a MVCNN or is it simply not supported?

    opened by danielbraun89 12
  • Analyzer setup times

    Analyzer setup times

    How come the instantiation times for the analyzer objects are so high, compared to the execution for given images, once the setup is done?

    Example for running the first three images in mnist_lrp.py

    Image 0: Input (0.0230s) Gradient*Input (0.1160s) LRP-Z (0.2516s) LRP-Z-IB (0.1634s) LRP-Epsilon (0.1742s) LRP-Epsilon-IB (0.1694s) LRP-W-Square (0.1779s) LRP-Flat (0.1840s) LRP-A2B1 (0.6371s) LRP-A2B1-IB (0.5809s) LRP-A1B0 (0.5755s) LRP-A1B0-IB (0.5152s) LRP-ZPlus (0.5752s) LRP-ZPlusFast (0.3181s)
    Image 1: Input (0.0005s) Gradient*Input (0.0016s) LRP-Z (0.0014s) LRP-Z-IB (0.0018s) LRP-Epsilon (0.0018s) LRP-Epsilon-IB (0.0016s) LRP-W-Square (0.0017s) LRP-Flat (0.0016s) LRP-A2B1 (0.0027s) LRP-A2B1-IB (0.0024s) LRP-A1B0 (0.0024s) LRP-A1B0-IB (0.0019s) LRP-ZPlus (0.0021s) LRP-ZPlusFast (0.0018s)
    Image 2: Input (0.0005s) Gradient*Input (0.0014s) LRP-Z (0.0018s) LRP-Z-IB (0.0015s) LRP-Epsilon (0.0017s) LRP-Epsilon-IB (0.0016s) LRP-W-Square (0.0013s) LRP-Flat (0.0014s) LRP-A2B1 (0.0026s) LRP-A2B1-IB (0.0027s) LRP-A1B0 (0.0024s) LRP-A1B0-IB (0.0020s) LRP-ZPlus (0.0024s) LRP-ZPlusFast (0.0019s)
    [...]
    

    Same for imagenet_lrp.py . Model is VGG16

    Image 0: Input (0.0193s) Gradient*Input (0.3270s) LRP-Z (2.0532s) LRP-Z-IB (1.9581s) LRP-Epsilon (2.9357s) LRP-Epsilon-IB (3.3425s) LRP-W-Square (3.3343s) LRP-Flat (3.5949s) LRP-A2B1 (13.3973s) LRP-A2B1-IB (15.5360s) LRP-A1B0 (21.1701s) LRP-A1B0-IB (21.2032s) LRP-ZPlus (23.4144s) LRP-ZPlusFast (12.3629s)
    Image 1: Input (0.0007s) Gradient*Input (0.0152s) LRP-Z (0.0228s) LRP-Z-IB (0.0224s) LRP-Epsilon (0.0236s) LRP-Epsilon-IB (0.0229s) LRP-W-Square (0.0230s) LRP-Flat (0.0230s) LRP-A2B1 (0.0651s) LRP-A2B1-IB (0.0637s) LRP-A1B0 (0.0375s) LRP-A1B0-IB (0.0360s) LRP-ZPlus (0.0358s) LRP-ZPlusFast (0.0214s)
    Image 2: Input (0.0010s) Gradient*Input (0.0147s) LRP-Z (0.0216s) LRP-Z-IB (0.0215s) LRP-Epsilon (0.0220s) LRP-Epsilon-IB (0.0217s) LRP-W-Square (0.0209s) LRP-Flat (0.0207s) LRP-A2B1 (0.0605s) LRP-A2B1-IB (0.0603s) LRP-A1B0 (0.0361s) LRP-A1B0-IB (0.0350s) LRP-ZPlus (0.0348s) LRP-ZPlusFast (0.0213s)
     [...]
    

    Once the (computational graph ? of the) analyzer has been built, excution times are low. What is causing this? Is it the copying of the layer params? Can we minimize that time?

    Edit: readability

    opened by sebastian-lapuschkin 11
  • All layer names should be unique.

    All layer names should be unique.

    I tested the package methods with VGG16 and it works. But, I have tested it with trained InceptionV3 model constructed in separate train script as following:

    base_model = InceptionV3(include_top=False, weights='imagenet',input_shape = (299,299,3))
    x = base_model.output
    x = GlobalAveragePooling2D()(x)
    x = Dense(1024, activation='relu')(x)
    x = Dropout(0.2)(x)
    predictions = Dense(nbr_classes, activation='softmax')(x)
    model = Model(input=base_model.input, output=predictions)
    

    And loaded in visualisation script as following (I based on imagenet_compare_methods.ipynb) :

    model_path = 'path_of_model'
    model_weights_path = 'path_of_model_weights'
    model = load_model(model_path)
    model.load_weights(model_weights_path)
    model = keras.models.Model(model.inputs, model.outputs)
    model.compile(optimizer="adam", loss="categorical_crossentropy")
    preprocess_f=keras.applications.inception_v3.preprocess_input
    color_coding="RGB"
    channels_first = keras.backend.image_data_format() == "channels_first"
    color_conversion = "BGRtoRGB" if color_coding == "BGR" else None
    input_range = (-1,1)
    noise_scale = (input_range[1]-input_range[0]) * 0.1
    

    I got the following error with iutils.keras.graph.model_wo_softmax function:

    Traceback (most recent call last):
      File "imagenet_compare_methods_incepV3.py", line 91, in <module>
        model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 312, in model_wo_softmax
        name=model.name)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
        return func(*args, **kwargs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 93, in __init__
        self._init_graph_network(*args, **kwargs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 231, in _init_graph_network
        self.inputs, self.outputs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 1455, in _map_graph_network
        ' times in the model. '
    ValueError: The name "dense_1" is used 2 times in the model. All layer names should be unique.
    

    To handle this error I have modified the names of my dense layers and I added this code:

    model.layers[-3].name='dense1'
    model.layers[-1].name='dense_final'
    

    These methodes [Input, Gradient] work proprly and SmoothGrad works but with out of memory warning. But with Guided Backprop I got an another error :

    Traceback (most recent call last):
      File "imagenet_compare_methods_incepV3.py", line 133, in <module>
        a = analyzer.analyze(x_pp)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 479, in analyze
        self.create_analyzer_model()
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 443, in create_analyzer_model
        outputs=analysis_outputs+debug_outputs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
        return func(*args, **kwargs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 93, in __init__
        self._init_graph_network(*args, **kwargs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 231, in _init_graph_network
        self.inputs, self.outputs)
      File "/home_nfs/brahimim/.conda/envs/mohammed.tf.1.9.0/lib/python3.6/site-packages/keras/engine/network.py", line 1455, in _map_graph_network
        ' times in the model. '
    ValueError: The name "activation_1" is used 2 times in the model. All layer names should be unique.
    

    Please help me to solve this issue.

    opened by Tahedi1 10
  • Pixelflipping

    Pixelflipping

    In the tool submodule adding a pixelflipping evaluation.

    Constructor inputs:

    • model, analyzer(s), function to merge attribution region (e.g., non-overlapping max-pooling), function to "blur" region, size of regions, number of steps.
    • evaluate (generator+normal interface):
      • do all analyzations, "blur", use fit_generator of model to get new predictions.
      • output, matrix: (analyzer, sample, number of steps) output.

    Multi-gpu?

    enhancement ForFirstRelease methods 
    opened by albermax 10
  • Exception: This is not supposed to happen! ERROR with ResNet50 transfer learning

    Exception: This is not supposed to happen! ERROR with ResNet50 transfer learning

    Hello, I was trying to use innvestigate in combination with a finetuned Resnet50 model which I created in keras, by taking off the last layer and adding a dense layer with sigmoid activation function. I loaded the model and created the analyzer according to tutorial, the problem is when I try to analyze a given image, I run into an Exception that only states that it is not supposed to happen. Maybe I am using the analyzer in a wrong way or maybe it doesn't work with my model. This my model:

    resnet = ResNet50(
            include_top = False,
            weights = 'imagenet',
            pooling = "avg",
    )
    
    #freeze the  layers
    for layer in resnet.layers:
            layer.trainable = True
    
    output_layer = layers.Dense(1, activation = "sigmoid")
    
    model = keras.models.Sequential([
            resnet, output_layer
        ])
    

    and this is the analayzer:

    model = keras.models.load_model("resnet50_model.h5")
    model.summary()
    
    image_height = image_width = 224
    batch_size = 32
    
    #Load test data
    test_datagen = ImageDataGenerator(rescale = 1.0/255)
    test_gen = test_datagen.flow_from_directory(
            "data/test/",
            target_size = (image_height, image_width),
            batch_size = batch_size,
            color_mode = "rgb",
            class_mode = "binary",
            shuffle = True,
            seed=1
    )
    
    #Select the first test image 
    image = x[1]
    
    plt.imshow(image, interpolation = "nearest")
    plt.show()
    
    #Take off the output layer
    model_wo_ol = iutils.keras.graph.model_wo_softmax(model)
    
    #Create gradient analyzer
    gradient_analyzer = innvestigate.create_analyzer("gradient", model_wo_ol)
    
    #Apply gradient analyzer
    gradient_analysis = gradient_analyzer.analyze(image)
    
    # Aggregate along color channels and normalize to [-1, 1]
    gradient_analysis = gradient_analysis.sum(axis=np.argmax(np.asarray(gradient_analysis.shape) == 3))
    gradient_analysis /= np.max(np.abs(gradient_analysis))
    # Plot and save image
    plt.imshow(gradient_analysis[0], cmap="seismic", clim=(-1, 1))
    plt.axis('off')
    plt.show()
    plt.savefig("gadient.png")
    

    that gives the following tracetrack: Screenshot from 2020-12-10 15-40-01

    Thank you.

    opened by saberAL 9
  • iNNvestigate 2.0 planning

    iNNvestigate 2.0 planning

    I suggest we approach the planning in the following way:

    • Define some high level stakeholder requirements.
    • Define more concrete Software requirements.
    • Develop the architecture and interfaces that would address those SW requirements.
    • Create a roadmap to know in which order we should do things.

    I know many things are "trivial", but IMHO it is good to have them listed and in mind when making decisions. Also compared to coding the whole package, planning it takes not that much time. :-)

    Also, those requirements change over time. The idea is to then update accordingly the derived requirements/architecture/interfaces/implementations.


    Stakeholder requirements

    Stakeholder is someone how will interact with this package in some way. Such requirements give a broad direction to the planning and are use case oriented. Mind stakeholders are more of a role than real persons - one person can have several roles.

    • Applying user: Someone who applies explanation methods w/o deep background knowledge and interest how everything works.
      • [SR 1.1] An applying user wants to be able easily setup and apply the software to standard neural network architectures.
      • [SR 1.2] An applying user wants to have a broad selection of the most important explanation methods (for neural networks).
      • [SR 1.3] An applying user wants to trust the SW to do the right thing.
    • Developing user: Someone who wants to develop/extend explanation methods.
      • [SR 2.1] A developing user wants to be able to easily integrate new methods and extend/modify existing ones.
    • Developers: Someone who actively works on the package.
      • [SR 3.1] A developer wants to be able to navigate the code easily and efficiently.
      • [SR 3.2] A developer wants to have a best practice and efficient workflows.

    Software requirements

    SW requirements are software oriented and typically derive from stakeholder requirements.

    • Derived from SR 1.1:
      • [SWR 1.11]: An easy to read and follow documentation/tutorials.
      • [SWR 1.1.2]: Works out of the box with (compatible) Keras models in a sensible way/with good default configurations.
      • [SWR 1.1.3]: Can be installed via pip.
      • [SWR 1.1.4]: Works with Python 3.6+
      • [SWR 1.1.5]: Works with TF/tf.keras 2.0+
      • [SWR 1.1.6]: The code runs quickly.
      • [SWR 1.1.7]: It is possible to save and export analysis models with tf.saved_models.
      • [SWR 1.1.8]: It is possible to choose the neuron(s) to analyze: index, max output, all, mask (for example for segmentation models).
    • Derived from SR 1.2:
      • [SWR 1.2.1]: LRP is implemented.
      • [SWR 1.2.2]: Smoothgrad is implemented.
      • [SWR 1.2.3]: Integrated gradients is implemented.
      • [SWR 1.2.4]: Vargard is implemented.
    • Derived from SR 1.3:
      • [SWR 1.3.1]: Code is reviewed by another developer before merging.
    • Derived from SR 2.1:
      • [SWR 2.1.1]: Clear interface between xyz and abc.
    • Derived from SR 3.1:
      • [SWR 3.1.1]: Use and enforcement of codings style xyz.
      • [SWR 3.1.2]: Use of automatic CI pipeline.
    • Derived from SR 3.2:
      • [SWR 3.2.1]: Use of a logger.

    Architecture & interfaces

    • Application interface:
      • Stick to sklearn fit/analyze pattern?
      • Stick to Keras models?
    • Architecture of a analyzer:
      • Divide preprocessing wrappers and analyzer functionality?
    • Algorithm:
      • Three steps: 1. check/remap/canonize network, 2. map to specific backward rules, 3. apply/build backward network
        • What should be the interfaces/invariants between those steps?
        • Consequently is an analyzer a collection of check/remap/canonize "rules" as well as "backward" rules?
    • Testing:
      • Unit tests; testing of single backward steps.
      • "Snapshot testing"?
      • Visual testing with notebooks?

    Open questions:

    • How to work on this content? Should we move to Google Docs or should I update the descriptions based on your inputs?

    Todos:

    • Checking what captum people are doing and how they are approaching this?
    1. Settle on a first set of requirements.
    2. Develop the architecture and interfaces.
    3. Develop a roadmap/order of implementations based on that.
    opened by albermax 9
  • LRP: AlphaBeta and Zplus

    LRP: AlphaBeta and Zplus

    I have found a bug in the code for the AlphaBeta rules and also the Zplus rule.

    The current code creates copies of the layer's weight and thesholds it at zero into positive and negative weight arrays. However, AlphaBeta and also Zplus require forward transformations thresholded at zero, which, with the current implementation, requires layer inputs x to be greater than or equal to zero.

    For a correct implementation of both rules (Zplus can inherit from AlphaBeta and realize alpha=1, beta=0), it would suffice to copy the layer's weights once, but then compute two forward passes which need to be thresholded. Does this interfere with the computation of the backward gradient?

    Also, for correctly considering the positive and negative forward contributions originating from the bias, the bias needs to be thresholded as well.

    Please have a look at (this)[https://github.com/sebastian-lapuschkin/lrp_toolbox/blob/python-wip/python/modules/linear.py], lines 219+, which implements the rules using numpy. Can this (conveniently) be fixed, without negatively interfering with the gradient computations?

    opened by sebastian-lapuschkin 9
  • [BUG] Remove `analyzer.fit()` from all example files. All example files using `analyzer.fit` method are broken

    [BUG] Remove `analyzer.fit()` from all example files. All example files using `analyzer.fit` method are broken

    Remove analyzer.fit() from all example files. All example files using analyzer.fit method are broken

    I was running example file examples/mnist_compare_methods.ipynb and I got error that analyzer.fit method doesn't exists and it was removed in this release [https://github.com/albermax/innvestigate/releases/tag/2.0.1]

    triage 
    opened by Rstar1998 0
  • Bump certifi from 2022.6.15.2 to 2022.12.7

    Bump certifi from 2022.6.15.2 to 2022.12.7

    Bumps certifi from 2022.6.15.2 to 2022.12.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump tensorflow from 2.10.0 to 2.10.1

    Bump tensorflow from 2.10.0 to 2.10.1

    Bumps tensorflow from 2.10.0 to 2.10.1.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.10.1

    Release 2.10.1

    This release introduces several vulnerability fixes:

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.10.1

    This release introduces several vulnerability fixes:

    Release 2.9.3

    This release introduces several vulnerability fixes:

    ... (truncated)

    Commits
    • fdfc646 Merge pull request #58581 from tensorflow-jenkins/version-numbers-2.10.1-4527
    • 319f094 Update version numbers to 2.10.1
    • 7c857b8 Merge pull request #58475 from tensorflow-jenkins/relnotes-2.10.1-6649
    • 6f133da Update RELEASE.md
    • 3982264 Merge pull request #58573 from tensorflow/r2.10-f856d02e532
    • f425d38 Merge pull request #58571 from tensorflow/r2.10-7b174a0f2e4
    • dbe4291 Merge pull request #58569 from tensorflow/r2.10-216525144ee
    • 965517a Merge pull request #58565 from tensorflow/r2.10-af4a6a3c8b9
    • c09738e Merge pull request #58564 from tensorflow/r2.10-9f03a9d3baf
    • 3da111c Merge pull request #58561 from tensorflow/r2.10-8310bf8dd18
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump pillow from 9.2.0 to 9.3.0

    Bump pillow from 9.2.0 to 9.3.0

    Bumps pillow from 9.2.0 to 9.3.0.

    Release notes

    Sourced from pillow's releases.

    9.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    9.3.0 (2022-10-29)

    • Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

    • Initialize libtiff buffer when saving #6699 [radarhere]

    • Inline fname2char to fix memory leak #6329 [nulano]

    • Fix memory leaks related to text features #6330 [nulano]

    • Use double quotes for version check on old CPython on Windows #6695 [hugovk]

    • Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

    • Fixed set_variation_by_name offset #6445 [radarhere]

    • Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

    • Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

    • Added ExifTags enums #6630 [radarhere]

    • Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

    • Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

    • Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

    • Added GPS TIFF tag info #6661 [radarhere]

    • Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

    • Do not attempt normalization if mode is already normal #6644 [radarhere]

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • [BUG] `AnalyzerNetworkBase` analyzers error when using `BatchNormalization` layers

    [BUG] `AnalyzerNetworkBase` analyzers error when using `BatchNormalization` layers

    On iNNvestigate v2.0.1, creating an analyzer inheriting from AnalyzerNetworkBase errors when the model contains a BatchNormalization layer, e.g.:

    tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'dense_2_input' with dtype float and shape [?,50]
    

    This might be due to batch normalisation layers keeping moving averages of the mean and standard deviation of the training data, causing problems with the Keras history when reversing the computational graph in iNNvestigate's create_analyzer_model.

    Minimal example reproducing the issue

    import numpy as np
    import tensorflow as tf
    from keras.layers import BatchNormalization, Dense
    from keras.models import Sequential
    
    import innvestigate
    
    tf.compat.v1.disable_eager_execution()
    
    input_shape = (50,)
    x = np.random.rand(100, *input_shape)
    y = np.random.rand(100, 2)
    
    model1 = Sequential()
    model1.add(Dense(10, input_shape=input_shape))
    model1.add(Dense(2))
    
    model2 = Sequential()
    model2.add(Dense(10, input_shape=input_shape))
    model2.add(BatchNormalization())
    model2.add(Dense(2))
    
    
    def run_analysis(model):
        model.compile(optimizer="adam", loss="mse")
        model.fit(x, y, epochs=10, verbose=0)
    
        analyzer = innvestigate.create_analyzer("gradient", model)
        analyzer.analyze(x)
    
    
    print("Model without BatchNormalization:")  # passes
    run_analysis(model1)
    print("Model with BatchNormalization:")     # errors
    run_analysis(model2)
    

    Full stacktrace

    Model with BatchNormalization:
    /Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/tensorflow/python/client/session.py:1480: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      ret = tf_session.TF_SessionRunCallable(self._session._session,
    Traceback (most recent call last):
      File "/Users/funks/Developer/innvestigate-issues/open/issue_238_v3", line 35, in <module>
        run_analysis(model2)
      File "/Users/funks/Developer/innvestigate-issues/open/issue_238_v3", line 29, in run_analysis
        analyzer.analyze(x)
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/innvestigate/analyzer/network_base.py", line 250, in analyze
        self.create_analyzer_model()
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/innvestigate/analyzer/network_base.py", line 196, in create_analyzer_model
        self._analyzer_model = kmodels.Model(
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/tensorflow/python/training/tracking/base.py", line 629, in _method_wrapper
        result = method(self, *args, **kwargs)
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/functional.py", line 146, in __init__
        self._init_graph_network(inputs, outputs)
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/tensorflow/python/training/tracking/base.py", line 629, in _method_wrapper
        result = method(self, *args, **kwargs)
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/functional.py", line 181, in _init_graph_network
        base_layer_utils.create_keras_history(self._nested_outputs)
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/base_layer_utils.py", line 175, in create_keras_history
        _, created_layers = _create_keras_history_helper(tensors, set(), [])
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/base_layer_utils.py", line 253, in _create_keras_history_helper
        processed_ops, created_layers = _create_keras_history_helper(
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/base_layer_utils.py", line 253, in _create_keras_history_helper
        processed_ops, created_layers = _create_keras_history_helper(
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/base_layer_utils.py", line 253, in _create_keras_history_helper
        processed_ops, created_layers = _create_keras_history_helper(
      [Previous line repeated 3 more times]
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/engine/base_layer_utils.py", line 251, in _create_keras_history_helper
        constants[i] = backend.function([], op_input)([])
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/keras/backend.py", line 4275, in __call__
        fetched = self._callable_fn(*array_vals,
      File "/Users/funks/Library/Caches/pypoetry/virtualenvs/innvestigate-issues-W0iScZgu-py3.10/lib/python3.10/site-packages/tensorflow/python/client/session.py", line 1480, in __call__
        ret = tf_session.TF_SessionRunCallable(self._session._session,
    tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'dense_2_input' with dtype float and shape [?,50]
             [[{{node dense_2_input}}]]
    
    bug 
    opened by adrhill 0
Releases(2.0.1)
  • 2.0.1(Sep 14, 2022)

  • 2.0.0(Aug 1, 2022)

    iNNvestigate for TensorFlow 2

    What's Changed

    • Format code by @adrhill in https://github.com/albermax/innvestigate/pull/247
    • Switch setup to Poetry by @adrhill in https://github.com/albermax/innvestigate/pull/257
    • Update CI by @adrhill in https://github.com/albermax/innvestigate/pull/258
    • Add issue templates by @adrhill in https://github.com/albermax/innvestigate/pull/259
    • Update backend by @adrhill in https://github.com/albermax/innvestigate/pull/263
    • Update analyzer init and serialization by @adrhill in https://github.com/albermax/innvestigate/pull/266
    • iNNvestigate 2.0 by @adrhill in https://github.com/albermax/innvestigate/pull/277
    • Update NumPy lower bound to 1.22, drop Python 3.7 support by @adrhill in https://github.com/albermax/innvestigate/pull/280

    New Contributors

    • @adrhill made their first contribution in https://github.com/albermax/innvestigate/pull/247

    Full Changelog: https://github.com/albermax/innvestigate/compare/1.0.8...2.0.0

    Source code(tar.gz)
    Source code(zip)
    innvestigate-2.0.0-py3-none-any.whl(65.67 KB)
    innvestigate-2.0.0.tar.gz(59.85 KB)
Owner
Maximilian Alber
Maximilian Alber
Data from "Datamodels: Predicting Predictions with Training Data"

Data from "Datamodels: Predicting Predictions with Training Data" Here we provid

Madry Lab 51 Dec 9, 2022
Machine Learning toolbox for Humans

Reproducible Experiment Platform (REP) REP is ipython-based environment for conducting data-driven research in a consistent and reproducible way. Main

Yandex 663 Dec 31, 2022
Python 3.6+ toolbox for submitting jobs to Slurm

Submit it! What is submitit? Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster. It basically wraps

Facebook Incubator 768 Jan 3, 2023
TorchDrug is a PyTorch-based machine learning toolbox designed for drug discovery

A powerful and flexible machine learning platform for drug discovery

MilaGraph 1.1k Jan 8, 2023
A Python implementation of the Robotics Toolbox for MATLAB

Robotics Toolbox for Python A Python implementation of the Robotics Toolbox for MATLAB® GitHub repository Documentation Wiki (examples and details) Sy

Peter Corke 1.2k Jan 7, 2023
PyPOTS - A Python Toolbox for Data Mining on Partially-Observed Time Series

A python toolbox/library for data mining on partially-observed time series, supporting tasks of forecasting/imputation/classification/clustering on incomplete multivariate time series with missing values.

Wenjie Du 179 Dec 31, 2022
High performance implementation of Extreme Learning Machines (fast randomized neural networks).

High Performance toolbox for Extreme Learning Machines. Extreme learning machines (ELM) are a particular kind of Artificial Neural Networks, which sol

Anton Akusok 174 Dec 7, 2022
Educational python for Neural Networks, written in pure Python/NumPy.

Educational python for Neural Networks, written in pure Python/NumPy.

null 127 Oct 27, 2022
An easier way to build neural search on the cloud

Jina is geared towards building search systems for any kind of data, including text, images, audio, video and many more. With the modular design & multi-layer abstraction, you can leverage the efficient patterns to build the system by parts, or chaining them into a Flow for an end-to-end experience.

Jina AI 17k Jan 1, 2023
neurodsp is a collection of approaches for applying digital signal processing to neural time series

neurodsp is a collection of approaches for applying digital signal processing to neural time series, including algorithms that have been proposed for the analysis of neural time series. It also includes simulation tools for generating plausible simulations of neural time series.

NeuroDSP 224 Dec 2, 2022
A concept I came up which ditches the idea of "layers" in a neural network.

Dynet A concept I came up which ditches the idea of "layers" in a neural network. Install Copy Dynet.py to your project. Run the example Install matpl

Anik Patel 4 Dec 5, 2021
Neural Ensemble Search for Performant and Calibrated Predictions

Neural Ensemble Search Introduction This repo contains the code accompanying the paper: Neural Ensemble Search for Performant and Calibrated Predictio

AutoML-Freiburg-Hannover 26 Dec 12, 2022
🔮 Execution time predictions for deep neural network training iterations across different GPUs.

Habitat: A Runtime-Based Computational Performance Predictor for Deep Neural Network Training Habitat is a tool that predicts a deep neural network's

Geoffrey Yu 44 Dec 27, 2022
Using / reproducing ACD from the paper "Hierarchical interpretations for neural network predictions" 🧠 (ICLR 2019)

Hierarchical neural-net interpretations (ACD) ?? Produces hierarchical interpretations for a single prediction made by a pytorch neural network. Offic

Chandan Singh 111 Jan 3, 2023
QTool: A Low-bit Quantization Toolbox for Deep Neural Networks in Computer Vision

This project provides abundant choices of quantization strategies (such as the quantization algorithms, training schedules and empirical tricks) for quantizing the deep neural networks into low-bit counterparts.

Monash Green AI Lab 51 Dec 10, 2022
Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

Complex-Valued Neural Networks (CVNN) Done by @NEGU93 - J. Agustin Barrachina Using this library, the only difference with a Tensorflow code is that y

youceF 1 Nov 12, 2021
Lime: Explaining the predictions of any machine learning classifier

lime This project is about explaining what machine learning classifiers (or models) are doing. At the moment, we support explaining individual predict

Marco Tulio Correia Ribeiro 10.3k Dec 29, 2022
naked is a Python tool which allows you to strip a model and only keep what matters for making predictions.

naked is a Python tool which allows you to strip a model and only keep what matters for making predictions. The result is a pure Python function with no third-party dependencies that you can simply copy/paste wherever you wish.

Max Halford 24 Dec 20, 2022
A library for debugging/inspecting machine learning classifiers and explaining their predictions

ELI5 ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions. It provides support for the following m

null 2.6k Dec 30, 2022