Hello!

When generating counterfactuals using CLUE counterfactual generator, the resulting counterfactuals in the dataframe are ordered using a RangeIndex instead of the original indexing found in the factuals. This is a problem when the factuals' indices are not ordered as a range index, e.x. by using .sample(n). This can be seen by running this code:

data_name = "compas"
model = MLModelCatalog(dataset, "ann", backend="pytorch")
model.train(...)

hyperparams = {...}
cl = Clue(dataset, model, hyperparams)

wa = Wachter(model, {...})

factuals = predict_negative_instances(model, dataset._df).sample(10)

cl_counterfactuals = cl.get_counterfactuals(factuals)
wa_counterfactuals = wa.get_counterfactuals(factuals)

display(factuals.index)
display(cl_counterfactuals.index)
display(wa_counterfactuals.index)

This yields:

Int64Index([4886, 4389, 2317, 797, 3154, 4685, 956, 3014, 99, 510], dtype='int64')
RangeIndex(start=0, stop=10, step=1)
Int64Index([4886, 4389, 2317, 797, 3154, 4685, 956, 3014, 99, 510], dtype='int64')

Now, if we try to benchmark the CLUE counterfactual generator:

benchmark = Benchmark(model, cl, factuals)
benchmark.run_benchmark()

we get:

ValueError: Can only compare identically-labeled DataFrame objects

Stemming from the constraint_violation check in carla\evaluation\violations.py

The counterfactuals get reordered in counterfactuals.check_counterfactuals for CLUE, FACE, GrowingSpheres and REVISE since they all pass in a list of counterfactuals as opposed to a pandas dataframe, so at least those methods can have problems with that.

One way of fixing this issue would be to pass an index attribute to check_counterfactuals and add a index=indices on line 32 to set the proper indices. Other would be to simply do cfs_df.index = factuals.index after the function is called to reorder the indices.

Since the counterfactuals now get new indices, and both dataframes are of the same size, restoring the ones from factuals shouldn't cause any further problems I can spot and will only improve the consistency across the recourse methods.

If any of the proposed fixes are acceptable, or there are other ways to fix the issue, I would be happy to perform them and be assigned to this issue. :)

Hi!

I've noticed some potentially wrong sparisty (L0/Distance_1) results due to some very small numbers.

When running Carla's benchmarking code for the cem-vae method and the first 10 test observations:

from carla.data.catalog import OnlineCatalog
from carla.models.catalog import MLModelCatalog
from carla.models.negative_instances import predict_negative_instances
from carla import Benchmark

import carla.recourse_methods.catalog as recourse_catalog

import torch

dataset = OnlineCatalog("adult")

torch.manual_seed(0)
n_test = 10
ml_model = MLModelCatalog(
        dataset, 
        model_type="ann", 
        load_online=False, 
        backend="pytorch"
    )

ml_model.train(
    learning_rate=0.002,
    epochs=20,
    batch_size=1024,
    hidden_size=[18, 9, 3],
    force_train=True, 
)

hyperparams = {
    "data_name": "adult",
    "batch_size": 1,
    "kappa": 0.1,
    "init_learning_rate": 0.01,
    "binary_search_steps": 9,
    "max_iterations": 100,
    "initial_const": 10,
    "beta": 0.9,
    "gamma": 1.0, # 0.0, #   1.0
    "mode": "PN",
    "num_classes": 2,
    "ae_params": {"hidden_layer": [20, 10, 7], "train_ae": True, "epochs": 5},
}

from tensorflow import Graph, Session

graph = Graph()
with graph.as_default():
    ann_sess = Session()
    with ann_sess.as_default():
        ml_model_sess = MLModelCatalog(dataset, "ann", "tensorflow")

        factuals_sess = predict_negative_instances(
            ml_model_sess, dataset.df
        )
        factuals_sess = factuals_sess.iloc[:n_test].reset_index(drop=True)

        cem = recourse_catalog.CEM(ann_sess, ml_model_sess, hyperparams)
        df_cfs = cem.get_counterfactuals(factuals_sess)
        benchmark = Benchmark(ml_model, cem, factuals_sess)

distances = benchmark.compute_distances()

distances.Distance_1[0] # equal to 5

I get that the first sparsity/Distance_1 is equal to 5. When printing our the factual and counterfactual for this test observation, I get that the two vectors are almost the same (the only difference is 'capital-gain').

The reason for this problem is that the distance_1 code looks something like this

import numpy as np

arr_f = ml_model.get_ordered_features(benchmark._factuals).to_numpy()
arr_cf = ml_model.get_ordered_features(
    benchmark._counterfactuals
).to_numpy()

delta = arr_f - arr_cf

d1 = np.sum(delta != 0, axis=1, dtype=np.float).tolist()

For the first observation, delta (the difference between the factual and the counterfactual) has really small (but not zero) numbers:

Which leads to a wrong calculation of d1.

Any suggestions on how to fix this delta/rounding problem?

Thanks!

Hi! Thanks for all the your work implementing a large range of counterfactual explanation methods!

I'm having problems running the Revise method and was hoping someone could point me in the right direction. Here is my code (I'm using the CARLA package master branch pushed April 20th, 2022):

from carla.data.catalog import OnlineCatalog
from carla.models.catalog import MLModelCatalog
from carla.models.negative_instances import predict_negative_instances
import carla.recourse_methods.catalog as recourse_catalog

data_name = "adult"
dataset = OnlineCatalog(data_name)

training_params = {"lr": 0.002, "epochs": 10, "batch_size": 1024, "hidden_size": [18, 9, 3]}

ml_model = MLModelCatalog(
    dataset, 
    model_type="ann", 
    load_online=False, 
    backend="pytorch"
)

ml_model.train(
    learning_rate=training_params["lr"],
    epochs=training_params["epochs"],
    batch_size=training_params["batch_size"],
    hidden_size=training_params["hidden_size"]
)

factuals = predict_negative_instances(ml_model, dataset.df)
test_factual = factuals.iloc[:5]

hyperparams = {
     "data_name": "adult",
        "lambda": 0.5,
        "optimizer": "adam",
        "lr": 0.1,
        "max_iter": 1000,
        "target_class": [0, 1],
        "binary_cat_features": True,
        "vae_params": {
            "layers": [13, 512, 256, 8],
            "train": True,
            "lambda_reg": 1e-6,
            "epochs": 5,
            "lr": 1e-3,
            "batch_size": 32,
        },
}

# This runs:
revise = recourse_catalog.Revise(ml_model, dataset, hyperparams)

# This gives me an error:
df_cfs = revise.get_counterfactuals(test_factual)

The last line of code gives me an IndexError: index 13 is out of bounds for dimension 1 with size 13. Any idea why this may be?

Thanks, Annabelle

Hello everyone,

I was trying to find here the implementation of the simple counterfactuals proposed by Wachter et al.? Could someone please help me in which of the files it is coded?

Thank you very very much! Cheers, Nina

Hello,

Many thanks for this great library.

I have a question: Does the current implementation of CCHVAE support categorical features?

If so, how should I set the number of nodes for the first layer (input layer) of the VAE (I mean for the vae_params['layers']? Should I set this value as the input length after one-hot encoding is applied?

Hi!

I ran all the dependent based methods using the CARLA master branch from April 22nd, 2022 for both the Adult and GMC data sets using an ANN with [18, 9, 3] layers, 20 epochs, and 1024 batch size (100 test observations) (AUC 0.90 and 0.86, respectively). Adult fixed 'age' and 'sex' and GMC fixed only 'age'. The hyperparameters for the dependent methods were specified as in https://github.com/carla-recourse/CARLA/blob/main/experimental_setup.yaml.

I used the Benchmarking code to test the violation/L0/L1/L2 metrics and was surprised by the results (these are the mean of violation/L0/L1/L2 across 100 test observations):

From my understanding, REVISE, C-CHVAE og CRUD methods all take into account fixed features. However, these three results had quite a large (and certainly not zero) violation rate.

In addition, C-CHVAE seems to produce almost identical counterfactuals for different test observations. This table

is from https://github.com/carla-recourse/CARLA/blob/chore/update_documentation/docs/source/notebooks/how_to_use_carla.ipynb (and I produced something similar) shows just how similar!

My question is whether you have experienced the same phenomenom. Do you have some results for REVISE/C-CHVAE/CRUD where the violation rate is close to zero or where C-CHVAE produces not identical counterfactuals?

Any help would be appreciated!

Annabelle

Hi there!

I have a few (3) questions about how to add a new recourse method

1. In your notebook example, there is:

def get_counterfactuals(self, factuals: pd.DataFrame):
      # this property is responsible to generate and output
      # encoded and scaled counterfactual examples
      # as pandas DataFrames
      return counterfactual_examples

I wonder what you mean exactly by encoded and scaled. Does that mean that they should follow the same encoding of factuals? Moreover, should there be exactly 1 counterfactual example per given factual? (I assume that factuals is a collection of points for which a counterfactual is needed).

2. I see you test the counterfactuals according to 4 distance functions. Do we have info on what distance function is used when the get_counterfactuals is called? I can imagine that you'd wish your recourse method to optimize for the distance that is ultimately used for evaluation.

3. Is there a way for the recourse method to know what is the range of variability of a feature? E.g., min and max for numerical features based on the training set, and the categorical possibilities for categorical features. Otherwise, I can imagine the black box model could be given an invalid input while searching for the counterfactuals (a too high or too small number or a category that does not exist).

Forgive me if this info is explained somewhere else and I missed it, in which case I'd kindly ask you to point me to it.

Thank you for all the work you put into the CARLA package. It's a great help when trying to compare different counterfactual methods! My question is about running the example code.

The part of posted example code is:

from carla import DataCatalog, MLModelCatalog from carla.recourse_methods import GrowingSpheres

load a catalog dataset

data_name = "adult" dataset = DataCatalog(data_name)

When I run the line "dataset = DataCatalog(data_name)", I got the error below: TypeError: Can't instantiate abstract class DataCatalog with abstract methods categorical, continuous, immutables, target

I am wondering do you know what is happening there or any way to solve that?

Appreciate your time in advance and really looking forward to your reply!

Best, Wenting

predict_negative_instances calls 'predict_label' which normalized the input, and calls 'model.predict'. If 'model.use_pipeline==True' then 'model.predict' again normalizes the data, resulting in double normalization.

The MLModelCatalog predict method has the following signature:

def predict(
    self, x: Union[np.ndarray, pd.DataFrame, torch.Tensor, tf.Tensor]
) -> Union[np.ndarray, pd.DataFrame, torch.Tensor, tf.Tensor]:

however if the MLModelCatalog pipeline is enabled then x is also input for

def perform_pipeline(self, df: pd.DataFrame) -> pd.DataFrame:

i.e. the predict function can take input types that are incompatible with the possible model settings.

I'm not an expert with pytorch and cuda but I did my best to add the .to(device) where necessary. CCHVAE and REViSE were much easier than CRUD (you'll see only a couple changes to vae.py, cchvae/model.py and revise/model.py). On the other hand, CRUD required a lot more work and there are maybe some redundant .to(device) (e.g., in losses.py or csvae.py) but at least the code works.

Since you don't have a GPU, might be difficult to test this out but perhaps you could check that the CPU version works?

I installed the package as recommend using pip install carla-recourse The version installed is carla-recourse==0.0.5 but I am not able to reproduce the benchmarking example. The error happen when importing evaluation module:

Hi,

Since the latest release of CARLA, I think there are some errors that have popped up.

The biggest problem is that (I believe) the dimension of the input layer of the VAE has to be adjusted for REViSE, CCHVAE, and CRUD if the immutable_mask contains at least one TRUE.

In addition, when running the methods on a GPU, there is some code that has to be adjusted (so far, I've only found problems for these three methods but I haven't tested all of them).

I'm not an expert on these methods, but can I do a pull request where I make changes to these methods such that they run again (on GPU)? Thanks,

Annabelle