Thank you for sharing your code. This is an impressive work.

I am trying to train FlowGMM in an unsupervised manner (no labels) on the MNIST dataset. I am using the script under experiments/train_flows/train_unsup.py. I am training with default parameters.

I am currently at epoch 10 / 100 and the loss is slowly decreasing (see attached image), but the generated samples do not look like hand-written digits (see attached). Is this issue resolved when the training is closer to being finished? Or do I need to run the training script with different arguments?

When I training the model for image classification, line 83 in file train_semisup_cons.py generates an error, shows "'RealNVP' object has no attribute 'module'". I checked your RealNVP file and didn't find attribute 'module' over there.

Hi, Thanks for the code. I am trying to run the experiments for CIFAR10, I downloaded the dataset via setup script. I installed the dependencies. But when I run the code with the example command given I am getting the following error.

Command I ran: python experiments/train_flows/train_semisup_cons.py --dataset=CIFAR10 --data_path=data/images/cifar/cifar10/by-image/ --label_path=data/labels/cifar10/4000_balanced_labels/10.txt --logdir=logdir --ckptdir=ckptdir --save_freq=500 --num_epochs=1501 --label_weight=3 --consistency_weight=1. --consistency_rampup=100 --lr=1e-4 --eval_freq=50

Output: 2020-10-23 15:47:34.626934: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.1/lib64 2020-10-23 15:47:34.626962: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

Traceback (most recent call last): File "experiments/train_flows/train_semisup_cons.py", line 259, in return_all_labels=args.acc_train_all_labels,)
File "/MYSYS_BASE_PATH/flowgmm/flow_ssl/data/ssl_data_utils.py", line 39, in make_ssl_data_loaders args, kwargs)
TypeError: make_ssl_cifar_data_loaders() got an unexpected keyword argument 'return_all_labels'


It would be great if you could help me out with this issue. Thanks in advance! Regards, Yashasvi

Hello, I cannot find the code in the repo to reproduce the NLP data results. I follow your instructions to preprocess the data and get the BERT embeddings. But it seems that the training command is missing and the code train_semisup_flowgmm_tabular.py got some unexpected error. I tried to adapt the code and command for tabular data to run Ag-News. It runs but the accuracy is quite low (~0.3). I guess I must have missed something important.

Could you please share the code or provide some guidance to reproduce the NLP data results?

I really appreciate your help and time!

Hi,

In the sample function in SSLGaussMixture, I'm confused in the following code

n_samples = sample_shape[0]
idx = np.random.choice(self.n_components, size=(n_samples, 1), p=F.softmax(self.weights))
all_samples = [g.sample(sample_shape) for g in self.gaussians]
samples = all_samples[0]
for i in range(self.n_components):
    mask = np.where(idx == i)
    samples[mask] = all_samples[i][mask]
return samples

I found that mask = np.where(idx == I) generates a 2 dimensional index (since idx is 2d), thus samples[mask] only selects single elements instead of rows. But I think samples[mask] should be replaced row by row accordingly? For example, with 128 dimensional means, I get mask=(array([9]), array([0])) and samples[mask]=tensor([1.4112]), but samples[mask] should be a 128d tensor instead.

Hello, I'm currently trying to use flowgmm model on my own nlp dataset. However, I found two problems quite confusing, do you mind clarifying: 1. Should all the data in my test dataset be manually labelled? Or I could just manually label parts of them? 2. I'm not sure how to generate this label file (LABELPATH — path to the label split generated by the data preparation scripts), should I put all the unlabelled and labelled data indices into this file?

Thank you so much for reading this issue!

When training the FlowGMM using "train_semisup_flowgmm_tabular.py", in the line 87: "thestudy.run(3,ordered=False)" , I get a typeError: makeTabularTrainer() got an unexpected keyword argument 'num_epochs'. I would appreciate it very much if you can help me deal with this error! @izmailovpavel

Hello, I confirm that some links in README of this repository don't work.

The example link that provided via README is follows. Text: The experiments on synthetic data are implemented in 'this ipython notebook'. Original Link: https://github.com/izmailovpavel/flow_ssl/blob/public/experiments/synthetic_data/synthetic.ipynb Link for revising: https://github.com/izmailovpavel/flowgmm/blob/public/experiments/synthetic_data/synthetic.ipynb

Thank you for reading this issue!

Hi,

I am wondering if it is relatively easy to use a custom tabular dataset using the synthetic notebook? I tried using a 15-dimensional dataset, but am running into a matrix multiplication error. Thanks in advance!

Sean

Hello, I've tried to train FlowGMM for the CIFAR10 dataset with the provided example code: python3 experiments/train_flows/train_semisup_cons.py --[arguments]

When I try to train with multiple GPUs with the option --gpu_ids (e.g., --gpu_ids=0,1), the following error comes out: RuntimeError: expected device cuda:1 but got device cuda:0.

Any suggestion for multiple GPU training? or Is the code only support single GPU training?

At: https://github.com/izmailovpavel/flowgmm/blob/422ff5dc15c5d2df6ada4787994d5915e11fcd4b/flow_ssl/distributions.py#L22-L25

You seem to insert a precision matrix, but reading the documentation:

https://github.com/pytorch/pytorch/blob/8b248af35d43c97d0e437f6f4ff0fbd4da5700c8/torch/distributions/multivariate_normal.py#L119

It seems like if you do not specify the matrix as a precision matrix, it will be assumed to be a covariance matrix (by argument order).

Is this intended?