Ladder network is a deep learning algorithm that combines supervised and unsupervised learning

Related tags

Game Development ladder
Overview

This repository contains source code for the experiments in a paper titled Semi-Supervised Learning with Ladder Networks by A Rasmus, H Valpola, M Honkala, M Berglund, and T Raiko.

Required libraries

Install Theano, Blocks Stable 0.2, Fuel Stable 0.2

Refer to the Blocks installation instructions for details but use tag v0.2 instead. Something along:

pip install git+git://github.com/mila-udem/[email protected]
pip install git+git://github.com/mila-udem/[email protected]

Fuel comes with Blocks, but you need to download and convert the datasets. Refer to the Fuel documentation. One might need to rename the converted files.

fuel-download mnist
fuel-convert mnist --dtype float32
fuel-download cifar10
fuel-convert cifar10
Alternatively, one can use the environment.yml file that is provided in this repo to create an conda environment.
  1. First install anaconda from https://www.continuum.io/downloads. Then,
  2. conda env create -f environment.yml
  3. source activate ladder
  4. The environment should be good to go!

Models in the paper

The following commands train the models with seed 1. The reported numbers in the paper are averages over several random seeds. These commands use all the training samples for training (--unlabeled-samples 60000) and none are used for validation. This results in a lot of NaNs being printed during the trainining, since the validation statistics are not available. If you want to observe the validation error and costs during the training, use --unlabeled-samples 50000.

MNIST all labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,1,0.01,0.01,0.01,0.01,0.01 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_full
# Bottom
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_all_baseline
MNIST 100 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 5000,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_bottom
# Gamma
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0.5 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_100_baseline
MNIST 1000 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --f-local-noise-std 0.2 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,10 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_1000_baseline
MNIST 50 labels
# Full model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full
MNIST convolutional models
# Conv-FC
run.py train --encoder-layers convv:1000:26:1:1-convv:500:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_fc
# Conv-Small, Gamma
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0,0,0,1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1  -- mnist_100_conv_gamma
# Conv-Small, supervised baseline. Overfits easily, so keep training short.
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --f-local-noise-std 0.45 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_baseline
CIFAR models
# Conv-Large, Gamma
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-gauss --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,4.0 --num-epochs 70 --lrate-decay 0.86 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_gamma
# Conv-Large, supervised baseline. Overfits easily, so keep training short.
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-0 --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_baseline
Evaluating models with testset

After training a model, you can infer the results on a test set by performing the evaluate command. An example use after training a model:

./run.py evaluate results/mnist_all_bottom0
Comments
  • AttributeError: 'numpy.float32' object has no attribute 'owner'

    AttributeError: 'numpy.float32' object has no attribute 'owner'

    I am using linux and theano 0.9, python 2.7 (under linux). I get the following attribute error. Any help?

    Thanks

    THEANO_FLAGS='floatX=float32' python run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full /home/me/ladder/venv2/local/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") INFO:main:Logging into results/mnist_50_full1/log.txt INFO:main:== COMMAND LINE == INFO:main:run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full INFO:main:== PARAMETERS == INFO:main: zestbn : bugfix
    INFO:main: dseed : 1
    INFO:main: top_c : 1
    INFO:main: super_noise_std : 0.3
    INFO:main: batch_size : 100
    INFO:main: dataset : mnist
    INFO:main: valid_set_size : 10000
    INFO:main: num_epochs : 150
    INFO:main: whiten_zca : 0
    INFO:main: unlabeled_samples : 60000
    INFO:main: decoder_spec : ('gauss',)
    INFO:main: valid_batch_size : 100
    INFO:main: denoising_cost_x : (2000.0, 20.0, 0.1, 0.1, 0.1, 0.1, 0.1) INFO:main: f_local_noise_std : 0.3
    INFO:main: cmd : train
    INFO:main: act : relu
    INFO:main: lrate_decay : 0.67
    INFO:main: seed : 1
    INFO:main: lr : 0.002
    INFO:main: save_to : mnist_50_full
    INFO:main: save_dir : results/mnist_50_full1 INFO:main: commit : 78956cdfc59110b557a759621abf7d391a6f5796 INFO:main: contrast_norm : 0
    INFO:main: encoder_layers : ('1000', '500', '250', '250', '250', '10') INFO:main: labeled_samples : 50
    INFO:main:Using 0 examples for validation INFO:main.model:Encoder: clean, labeled INFO:main.model: 0: noise 0 INFO:main.model: f1: fc, relu, BN, noise 0.00, params 1000, dim (1, 28, 28) -> (1000,) INFO:main.model: f2: fc, relu, BN, noise 0.00, params 500, dim (1000,) -> (500,) INFO:main.model: f3: fc, relu, BN, noise 0.00, params 250, dim (500,) -> (250,) INFO:main.model: f4: fc, relu, BN, noise 0.00, params 250, dim (250,) -> (250,) INFO:main.model: f5: fc, relu, BN, noise 0.00, params 250, dim (250,) -> (250,) INFO:main.model: f6: fc, softmax, BN, noise 0.00, params 10, dim (250,) -> (10,) INFO:main.model:Encoder: corr, labeled INFO:main.model: 0: noise 0.3 INFO:main.model: f1: fc, relu, BN, noise 0.30, params 1000, dim (1, 28, 28) -> (1000,) INFO:main.model: f2: fc, relu, BN, noise 0.30, params 500, dim (1000,) -> (500,) INFO:main.model: f3: fc, relu, BN, noise 0.30, params 250, dim (500,) -> (250,) INFO:main.model: f4: fc, relu, BN, noise 0.30, params 250, dim (250,) -> (250,) INFO:main.model: f5: fc, relu, BN, noise 0.30, params 250, dim (250,) -> (250,) INFO:main.model: f6: fc, softmax, BN, noise 0.30, params 10, dim (250,) -> (10,) INFO:main.model:Decoder: z_corr -> z_est INFO:main.model: g6: gauss, denois 0.10, dim None -> (10,) INFO:main.model: g5: gauss, denois 0.10, dim (10,) -> (250,) INFO:main.model: g4: gauss, denois 0.10, dim (250,) -> (250,) INFO:main.model: g3: gauss, denois 0.10, dim (250,) -> (250,) INFO:main.model: g2: gauss, denois 0.10, dim (250,) -> (500,) INFO:main.model: g1: gauss, denois 20.00, dim (500,) -> (1000,) INFO:main.model: g0: gauss, denois 2000.00, dim (1000,) -> (1, 28, 28) INFO:main:Found the following parameters: [f_5_b, f_4_b, f_3_b, f_2_b, f_1_b, g_6_a5, f_6_c, f_6_b, g_6_a4, g_6_a3, g_6_a2, g_6_a1, g_6_a10, g_6_a9, g_6_a8, g_6_a7, g_6_a6, g_5_a5, g_5_a4, g_5_a3, g_5_a2, g_5_a1, g_5_a10, g_5_a9, g_5_a8, g_5_a7, g_5_a6, g_4_a5, g_4_a4, g_4_a3, g_4_a2, g_4_a1, g_4_a10, g_4_a9, g_4_a8, g_4_a7, g_4_a6, g_3_a5, g_3_a4, g_3_a3, g_3_a2, g_3_a1, g_3_a10, g_3_a9, g_3_a8, g_3_a7, g_3_a6, g_2_a5, g_2_a4, g_2_a3, g_2_a2, g_2_a1, g_2_a10, g_2_a9, g_2_a8, g_2_a7, g_2_a6, g_1_a5, g_1_a4, g_1_a3, g_1_a2, g_1_a1, g_1_a10, g_1_a9, g_1_a8, g_1_a7, g_1_a6, g_0_a5, g_0_a4, g_0_a3, g_0_a2, g_0_a1, g_0_a10, g_0_a9, g_0_a8, g_0_a7, g_0_a6, f_1_W, f_2_W, f_3_W, f_4_W, f_5_W, f_6_W, g_5_W, g_4_W, g_3_W, g_2_W, g_1_W, g_0_W] INFO:blocks.algorithms:Taking the cost gradient INFO:blocks.algorithms:The cost gradient computation graph is built INFO:main:Balancing 50 labels... INFO:main.nn:Batch norm parameters: f_1_bn_mean_clean, f_1_bn_var_clean, f_2_bn_mean_clean, f_2_bn_var_clean, f_3_bn_mean_clean, f_3_bn_var_clean, f_4_bn_mean_clean, f_4_bn_var_clean, f_5_bn_mean_clean, f_5_bn_var_clean, f_6_bn_mean_clean, f_6_bn_var_clean INFO:main:Balancing 50 labels... INFO:main.nn:Batch norm parameters: f_1_bn_mean_clean, f_1_bn_var_clean, f_2_bn_mean_clean, f_2_bn_var_clean, f_3_bn_mean_clean, f_3_bn_var_clean, f_4_bn_mean_clean, f_4_bn_var_clean, f_5_bn_mean_clean, f_5_bn_var_clean, f_6_bn_mean_clean, f_6_bn_var_clean INFO:blocks.main_loop:Entered the main loop /home/me/ladder/venv2/local/lib/python2.7/site-packages/pandas/core/generic.py:1101: PerformanceWarning: your performance may suffer as PyTables will pickle object types that it cannot map directly to c-types [inferred_type->mixed-integer,key->block0_values] [items->[0]]

    return pytables.to_hdf(path_or_buf, key, self, **kwargs) INFO:blocks.algorithms:Initializing the training algorithm ERROR:blocks.main_loop:Error occured during training.

    Blocks will attempt to run on_error extensions, potentially saving data, before exiting and reraising the error. Note that the usual after_training extensions will not be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately. Traceback (most recent call last): File "run.py", line 653, in if train(d) is None: File "run.py", line 502, in train main_loop.run() File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/main_loop.py", line 197, in run reraise_as(e) File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/utils/init.py", line 258, in reraise_as six.reraise(type(new_exc), new_exc, orig_exc_traceback) File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/main_loop.py", line 172, in run self.algorithm.initialize() File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/algorithms/init.py", line 128, in initialize self.inputs = ComputationGraph(update_values).inputs File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/graph/init.py", line 74, in init self._get_variables() File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/blocks/graph/init.py", line 125, in _get_variables inputs = graph.inputs(self.outputs) File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/theano/gof/graph.py", line 693, in inputs vlist = ancestors(variable_list, blockers) File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/theano/gof/graph.py", line 672, in ancestors dfs_variables = stack_search(deque(variable_list), expand, 'dfs') File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/theano/gof/graph.py", line 640, in stack_search expand_l = expand(l) File "/home/me/ladder/venv2/local/lib/python2.7/site-packages/theano/gof/graph.py", line 670, in expand if r.owner and (not blockers or r not in blockers): AttributeError: 'numpy.float32' object has no attribute 'owner'

    Original exception: AttributeError: 'numpy.float32' object has no attribute 'owner'

    opened by peymanr 11
  • I met a problem: fuel.config_parser.ConfigurationError: Configuration not set and no default provided: data_path.

    I met a problem: fuel.config_parser.ConfigurationError: Configuration not set and no default provided: data_path.

    wangxiao@GTX980:~/Desktop/ladder-master$ ./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") ERROR:main:Subprocess returned fatal: Not a git repository (or any of the parent directories): .git INFO:main:Logging into results/mnist_all_bottom0/log.txt INFO:main:== COMMAND LINE == INFO:main:./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom INFO:main:== PARAMETERS == INFO:main: zestbn : bugfix
    INFO:main: dseed : 1
    INFO:main: top_c : 1
    INFO:main: super_noise_std : 0.3
    INFO:main: batch_size : 100
    INFO:main: dataset : mnist
    INFO:main: valid_set_size : 10000
    INFO:main: num_epochs : 150
    INFO:main: whiten_zca : 0
    INFO:main: unlabeled_samples : 60000
    INFO:main: decoder_spec : ('gauss',)
    INFO:main: valid_batch_size : 100
    INFO:main: denoising_cost_x : (2000.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0) INFO:main: f_local_noise_std : 0.3
    INFO:main: cmd : train
    INFO:main: act : relu
    INFO:main: lrate_decay : 0.67
    INFO:main: seed : 1
    INFO:main: lr : 0.002
    INFO:main: save_to : mnist_all_bottom
    INFO:main: save_dir : results/mnist_all_bottom0 INFO:main: commit :
    INFO:main: contrast_norm : 0
    INFO:main: encoder_layers : ('1000', '500', '250', '250', '250', '10') INFO:main: labeled_samples : 60000
    Traceback (most recent call last): File "./run.py", line 651, in if train(d) is None: File "./run.py", line 405, in train in_dim, data, whiten, cnorm = setup_data(p, test_set=False) File "./run.py", line 240, in setup_data train_set = dataset_class("train") File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 36, in init super(MNIST, self).init(self.data_path, which_set, *kwargs) File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 40, in data_path return os.path.join(config.data_path, self.filename) File "/usr/local/lib/python2.7/dist-packages/fuel/config_parser.py", line 101, in getattr "provided: {}.".format(key)) fuel.config_parser.ConfigurationError: Configuration not set and no default provided: data_path. wangxiao@GTX980:~/Desktop/ladder-master$ ./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") ERROR:main:Subprocess returned fatal: Not a git repository (or any of the parent directories): .git INFO:main:Logging into results/mnist_all_gamma0/log.txt INFO:main:== COMMAND LINE == INFO:main:./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma INFO:main:== PARAMETERS == INFO:main: zestbn : bugfix
    INFO:main: dseed : 1
    INFO:main: top_c : 1
    INFO:main: super_noise_std : 0.3
    INFO:main: batch_size : 100
    INFO:main: dataset : mnist
    INFO:main: valid_set_size : 10000
    INFO:main: num_epochs : 150
    INFO:main: whiten_zca : 0
    INFO:main: unlabeled_samples : 60000
    INFO:main: decoder_spec : ('0', '0', '0', '0', '0', '0', 'gauss') INFO:main: valid_batch_size : 100
    INFO:main: denoising_cost_x : (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0) INFO:main: f_local_noise_std : 0.3
    INFO:main: cmd : train
    INFO:main: act : relu
    INFO:main: lrate_decay : 0.67
    INFO:main: seed : 1
    INFO:main: lr : 0.002
    INFO:main: save_to : mnist_all_gamma
    INFO:main: save_dir : results/mnist_all_gamma0 INFO:main: commit :
    INFO:main: contrast_norm : 0
    INFO:main: encoder_layers : ('1000', '500', '250', '250', '250', '10') INFO:main: labeled_samples : 60000
    Traceback (most recent call last): File "./run.py", line 651, in if train(d) is None: File "./run.py", line 405, in train in_dim, data, whiten, cnorm = setup_data(p, test_set=False) File "./run.py", line 240, in setup_data train_set = dataset_class("train") File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 36, in init super(MNIST, self).init(self.data_path, which_set, kwargs) File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 40, in data_path return os.path.join(config.data_path, self.filename) File "/usr/local/lib/python2.7/dist-packages/fuel/config_parser.py", line 101, in getattr "provided: {}.".format(key)) *__fuel.config_parser.ConfigurationError: Configuration not set and no default provided: data_path.     wangxiao@GTX980:~/Desktop/ladder-master$

    opened by wangxiao5791509 10
  • ValueError: setting an array element with a sequence.

    ValueError: setting an array element with a sequence.

    I'm trying to run one of the commands from the README file:

    THEANO_FLAGS='floatX=float32' ./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,1,0.01,0.01,0.01,0.01,0.01 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_full

    but keep getting the following error:

    Traceback (most recent call last):
  File "./run.py", line 651, in <module>
    if train(d) is None:
  File "./run.py", line 423, in train
    step_rule=Adam(learning_rate=ladder.lr))
  File "/usr/local/lib/python2.7/site-packages/blocks/algorithms/__init__.py", line 794, in __init__
    self.learning_rate = shared_floatx(learning_rate, "learning_rate")
  File "/usr/local/lib/python2.7/site-packages/blocks/utils/__init__.py", line 167, in shared_floatx
    return theano.shared(theano._asarray(value, dtype=dtype),
  File "/usr/local/lib/python2.7/site-packages/theano/misc/safe_asarray.py", line 33, in _asarray
    rval = numpy.asarray(a, dtype=dtype, order=order)
  File "/usr/local/lib/python2.7/site-packages/numpy/core/numeric.py", line 474, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

    If I look at the parameters passed to "array(...)" in line 474 of numeric.py, I see the following:

    a ->  learning_rate 
type(a) -> <class 'theano.tensor.sharedvar.ScalarSharedVariable'> 
dtype -> float64

    Could this be due to a version conflict of any of the involved libraries?

    opened by kleiba 6
  • ValueError: '['train']' split is not provided by this dataset. Available splits are (u'test', u'train').

    ValueError: '['train']' split is not provided by this dataset. Available splits are (u'test', u'train').

    Hi, I just setup the environment, and got a issue in my first running. Could you please show me how to solve this problem, many thanks!!!

    $ python run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full

    INFO:main:Logging into results/mnist_50_full6/log.txt INFO:main:== COMMAND LINE == INFO:main:run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full INFO:main:== PARAMETERS == INFO:main: zestbn : bugfix
    INFO:main: dseed : 1
    INFO:main: top_c : 1
    INFO:main: super_noise_std : 0.3
    INFO:main: batch_size : 100
    INFO:main: dataset : mnist
    INFO:main: valid_set_size : 10000
    INFO:main: num_epochs : 150
    INFO:main: whiten_zca : 0
    INFO:main: unlabeled_samples : 60000
    INFO:main: decoder_spec : ('gauss',)
    INFO:main: valid_batch_size : 100
    INFO:main: denoising_cost_x : (2000.0, 20.0, 0.1, 0.1, 0.1, 0.1, 0.1) INFO:main: f_local_noise_std : 0.3
    INFO:main: cmd : train
    INFO:main: act : relu
    INFO:main: lrate_decay : 0.67
    INFO:main: seed : 1
    INFO:main: lr : 0.002
    INFO:main: save_to : mnist_50_full
    INFO:main: save_dir : results/mnist_50_full6 INFO:main: commit : 5a8daa1760535ec4aa25c20c531e1cc31c76d911 INFO:main: contrast_norm : 0
    INFO:main: encoder_layers : ('1000', '500', '250', '250', '250', '10') INFO:main: labeled_samples : 50
    Traceback (most recent call last): File "run.py", line 656, in if train(d) is None: File "run.py", line 410, in train in_dim, data, whiten, cnorm = setup_data(p, test_set=False) File "run.py", line 245, in setup_data train_set = dataset_class(["train"]) File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 36, in init super(MNIST, self).init(self.data_path, which_set, **kwargs) File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/hdf5.py", line 146, in init "{}.".format(self.available_splits)) ValueError: '['train']' split is not provided by this dataset. Available splits are (u'test', u'train').

    opened by tongmuyuan 5
  • The image and the kernel must have the same type.inputs(float64), kerns(float32)

    The image and the kernel must have the same type.inputs(float64), kerns(float32)

    I got the following error when trying to run the cifar10 example:

    Traceback (most recent call last):
  File "./run.py", line 652, in <module>
    if train(d) is None:
  File "./run.py", line 410, in train
    ladder = setup_model(p)
  File "./run.py", line 181, in setup_model
    ladder.apply(x, y, x_only)
  File "/home/petteri/ladder/ladder.py", line 195, in apply
    clean = self.act.clean = encoder(input_concat, 'clean')
  File "/home/petteri/ladder/ladder.py", line 183, in encoder
    noise_std=noise)
  File "/home/petteri/ladder/ladder.py", line 349, in f
    z, output_size = self.f_conv(h, spec, in_dim, gen_id('W'))
  File "/home/petteri/ladder/ladder.py", line 450, in f_conv
    filter_size), border_mode=bm)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/tensor/nnet/conv.py", line 153, in conv2d
    return op(input, filters)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/gof/op.py", line 602, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/tensor/nnet/conv.py", line 655, in make_node
    "inputs(%s), kerns(%s)" % (_inputs.dtype, _kerns.dtype))
NotImplementedError: The image and the kernel must have the same type.inputs(float64), kerns(float32)

    Which got fixed by casting manually at line 180 of ladder.py:

    h = T.cast(h, 'float32')

    But this again later leads to TypeError:

    INFO:blocks.algorithms:Initializing the training algorithm
ERROR:blocks.main_loop:Error occured during training.

Blocks will attempt to run `on_error` extensions, potentially saving data, before exiting and reraising the error. Note that the usual `after_training` extensions will *not* be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately.
Traceback (most recent call last):
  File "./run.py", line 652, in <module>
    if train(d) is None:
  File "./run.py", line 500, in train
    main_loop.run()
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/blocks/main_loop.py", line 188, in run
    reraise_as(e)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/blocks/utils/__init__.py", line 225, in reraise_as
    six.reraise(type(new_exc), new_exc, orig_exc_traceback)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/blocks/main_loop.py", line 164, in run
    self.algorithm.initialize()
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/blocks/algorithms/__init__.py", line 224, in initialize
    self._function = theano.function(self.inputs, [], updates=all_updates)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/compile/function.py", line 322, in function
    output_keys=output_keys)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py", line 443, in pfunc
    no_default_updates=no_default_updates)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py", line 208, in rebuild_collect_shared
    raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=f_11_b, shared_var.type=TensorType(float32, vector), update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, vector))., If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.\n\nOriginal exception:\n\tTypeError: An update must have the same type as the original shared variable (shared_var=f_11_b, shared_var.type=TensorType(float32, vector), update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, vector))., If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

    Any thoughts on where it goes wrong?

    opened by petteriTeikari 5
  • NaN in V_C_class, V_E, V_C_de

    NaN in V_C_class, V_E, V_C_de

    I am trying to run this but got NaN errors when the program prints validation costs/accuracy. The command I ran is:

    THEANO_FLAGS='floatX=float32,device=gpu0,lib.cnmem=0.8' FUEL_DATA_PATH=. \
./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss \
--denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 \
--unlabeled-samples 60000 --seed 1 -- mnist_100_full

    Logger output:

    Using gpu device 0: GeForce GTX 1060 6GB (CNMeM is enabled with initial size: 80.0% of memory, cuDNN not available)
INFO:main:Logging into results/mnist_100_full5/log.txt
INFO:main:== COMMAND LINE ==
INFO:main:./run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full
INFO:main:== PARAMETERS ==
INFO:main: zestbn              : bugfix               
INFO:main: dseed               : 1                    
INFO:main: top_c               : 1                    
INFO:main: super_noise_std     : 0.3                  
INFO:main: batch_size          : 100                  
INFO:main: dataset             : mnist                
INFO:main: valid_set_size      : 10000                
INFO:main: num_epochs          : 150                  
INFO:main: whiten_zca          : 0                    
INFO:main: unlabeled_samples   : 60000                
INFO:main: decoder_spec        : ('gauss',)           
INFO:main: valid_batch_size    : 100                  
INFO:main: denoising_cost_x    : (1000.0, 10.0, 0.1, 0.1, 0.1, 0.1, 0.1) 
INFO:main: f_local_noise_std   : 0.3                  
INFO:main: cmd                 : train                
INFO:main: act                 : relu                 
INFO:main: lrate_decay         : 0.67                 
INFO:main: seed                : 1                    
INFO:main: lr                  : 0.002                
INFO:main: save_to             : mnist_100_full       
INFO:main: save_dir            : results/mnist_100_full5 
INFO:main: commit              : 6df36de9d9a6e69dd55ce0985ebbb489633d118e 
INFO:main: contrast_norm       : 0                    
INFO:main: encoder_layers      : ('1000', '500', '250', '250', '250', '10') 
INFO:main: labeled_samples     : 100                  
INFO:main:Using 0 examples for validation
INFO:main.model:Encoder: clean, labeled
INFO:main.model:  0: noise 0
INFO:main.model:  f1: fc, relu, BN, noise 0.00, params 1000, dim (1, 28, 28) -> (1000,)
INFO:main.model:  f2: fc, relu, BN, noise 0.00, params 500, dim (1000,) -> (500,)
INFO:main.model:  f3: fc, relu, BN, noise 0.00, params 250, dim (500,) -> (250,)
INFO:main.model:  f4: fc, relu, BN, noise 0.00, params 250, dim (250,) -> (250,)
INFO:main.model:  f5: fc, relu, BN, noise 0.00, params 250, dim (250,) -> (250,)
INFO:main.model:  f6: fc, softmax, BN, noise 0.00, params 10, dim (250,) -> (10,)
INFO:main.model:Encoder: corr, labeled
INFO:main.model:  0: noise 0.3
INFO:main.model:  f1: fc, relu, BN, noise 0.30, params 1000, dim (1, 28, 28) -> (1000,)
INFO:main.model:  f2: fc, relu, BN, noise 0.30, params 500, dim (1000,) -> (500,)
INFO:main.model:  f3: fc, relu, BN, noise 0.30, params 250, dim (500,) -> (250,)
INFO:main.model:  f4: fc, relu, BN, noise 0.30, params 250, dim (250,) -> (250,)
INFO:main.model:  f5: fc, relu, BN, noise 0.30, params 250, dim (250,) -> (250,)
INFO:main.model:  f6: fc, softmax, BN, noise 0.30, params 10, dim (250,) -> (10,)
INFO:main.model:Decoder: z_corr -> z_est
INFO:main.model:  g6:      gauss, denois 0.10, dim None -> (10,)
INFO:main.model:  g5:      gauss, denois 0.10, dim (10,) -> (250,)
INFO:main.model:  g4:      gauss, denois 0.10, dim (250,) -> (250,)
INFO:main.model:  g3:      gauss, denois 0.10, dim (250,) -> (250,)
INFO:main.model:  g2:      gauss, denois 0.10, dim (250,) -> (500,)
INFO:main.model:  g1:      gauss, denois 10.00, dim (500,) -> (1000,)
INFO:main.model:  g0:      gauss, denois 1000.00, dim (1000,) -> (1, 28, 28)
INFO:main:Found the following parameters: [f_6_W, f_5_b, f_5_W, f_4_b, f_4_W, f_3_b, f_3_W, f_2_b, f_2_W, f_1_b, f_1_W, g_6_a5, f_6_c, f_6_b, g_6_a4, g_6_a3, g_6_a2, g_6_a1, g_6_a10, g_6_a9, g_6_a8, g_6_a7, g_6_a6, g_5_a5, g_5_W, g_5_a4, g_5_a3, g_5_a2, g_5_a1, g_5_a10, g_5_a9, g_5_a8, g_5_a7, g_5_a6, g_4_a5, g_4_W, g_4_a4, g_4_a3, g_4_a2, g_4_a1, g_4_a10, g_4_a9, g_4_a8, g_4_a7, g_4_a6, g_3_a5, g_3_W, g_3_a4, g_3_a3, g_3_a2, g_3_a1, g_3_a10, g_3_a9, g_3_a8, g_3_a7, g_3_a6, g_2_a5, g_2_W, g_2_a4, g_2_a3, g_2_a2, g_2_a1, g_2_a10, g_2_a9, g_2_a8, g_2_a7, g_2_a6, g_1_a5, g_1_W, g_1_a4, g_1_a3, g_1_a2, g_1_a1, g_1_a10, g_1_a9, g_1_a8, g_1_a7, g_1_a6, g_0_a5, g_0_W, g_0_a4, g_0_a3, g_0_a2, g_0_a1, g_0_a10, g_0_a9, g_0_a8, g_0_a7, g_0_a6]
INFO:blocks.algorithms:Taking the cost gradient
INFO:blocks.algorithms:The cost gradient computation graph is built
INFO:main:Balancing 100 labels...
INFO:main.nn:Batch norm parameters: f_1_bn_mean_clean, f_1_bn_var_clean, f_2_bn_mean_clean, f_2_bn_var_clean, f_3_bn_mean_clean, f_3_bn_var_clean, f_4_bn_mean_clean, f_4_bn_var_clean, f_5_bn_mean_clean, f_5_bn_var_clean, f_6_bn_mean_clean, f_6_bn_var_clean
INFO:main:Balancing 100 labels...
INFO:main.nn:Batch norm parameters: f_1_bn_mean_clean, f_1_bn_var_clean, f_2_bn_mean_clean, f_2_bn_var_clean, f_3_bn_mean_clean, f_3_bn_var_clean, f_4_bn_mean_clean, f_4_bn_var_clean, f_5_bn_mean_clean, f_5_bn_var_clean, f_6_bn_mean_clean, f_6_bn_var_clean
INFO:blocks.main_loop:Entered the main loop
/home/mren/anaconda2/envs/ladder/lib/python2.7/site-packages/pandas/core/generic.py:939: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed-integer,key->block0_values] [items->[0]]

  return pytables.to_hdf(path_or_buf, key, self, **kwargs)
INFO:blocks.algorithms:Initializing the training algorithm
INFO:blocks.algorithms:The training algorithm is initialized
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data started
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data finished
INFO:main.utils:e 0, i 0:V_C_class nan, V_E nan, V_C_de nan nan nan nan nan nan nan
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data started
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data finished
INFO:main.utils:e 1, i 600:V_C_class nan, V_E nan, V_C_de nan nan nan nan nan nan nan, T_C_de 0.019 0.253 0.994 0.955 0.843 0.679 0.231, T_C_class 0.156
INFO:main.nn:Iter 1, lr 0.002000
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data started
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data finished
INFO:main.utils:e 2, i 1200:V_C_class nan, V_E nan, V_C_de nan nan nan nan nan nan nan, T_C_de 0.0103 0.135 0.975 0.9 0.422 0.2 0.0925, T_C_class 0.0308
INFO:main.nn:Iter 2, lr 0.002000
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data started
INFO:blocks.extensions.monitoring:Monitoring on auxiliary data finished
INFO:main.utils:e 3, i 1800:V_C_class nan, V_E nan, V_C_de nan nan nan nan nan nan nan, T_C_de 0.00909 0.123 0.97 0.863 0.239 0.131 0.0731, T_C_class 0.0143
INFO:main.nn:Iter 3, lr 0.002000
    opened by renmengye 3
  • Error during trainning model with CIFAR10

    Error during trainning model with CIFAR10

    `INFO:main.utils:e 0, i 0:V_C_class nan, V_E nan, V_C_de nan ERROR:blocks.main_loop:Error occured during training.

    Blocks will attempt to run on_error extensions, potentially saving data, before exiting and reraising the error. Note that the usual after_training extensions will not be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately. Traceback (most recent call last): File "run.py", line 660, in if train(d) is None: File "run.py", line 509, in train main_loop.run() File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/main_loop.py", line 197, in run reraise_as(e) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/utils/init.py", line 258, in reraise_as six.reraise(type(new_exc), new_exc, orig_exc_traceback) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/main_loop.py", line 183, in run while self._run_epoch(): File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/main_loop.py", line 232, in _run_epoch while self._run_iteration(): File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/main_loop.py", line 253, in _run_iteration self.algorithm.process_batch(batch) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/blocks/algorithms/init.py", line 287, in process_batch self._function(*ordered_batch) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/theano/compile/function_module.py", line 871, in call storage_map=getattr(self.fn, 'storage_map', None)) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op reraise(exc_type, exc_value, exc_trace) File "/home/julian/anaconda2/envs/ladder/lib/python2.7/site-packages/theano/compile/function_module.py", line 859, in call outputs = self.fn() ValueError: GpuElemwise. Input dimension mis-match. Input 2 (indices start at 0) has shape[2] == 6, but the output's size on that axis is 5. Apply node that caused the error: GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)](GpuJoin.0, CudaNdarrayConstant{[[[[ 0.30000001]]]]}, GpuReshape{4}.0, GpuDimShuffle{x,0,x,x}.0) Toposort index: 1535 Inputs types: [CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, (True, True, True, True)), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, (True, False, True, True))] Inputs shapes: [(200, 192, 5, 5), (1, 1, 1, 1), (200, 192, 6, 6), (1, 192, 1, 1)] Inputs strides: [(4800, 25, 5, 1), (0, 0, 0, 0), (6912, 36, 6, 1), (0, 1, 0, 0)] Inputs values: ['not shown', CudaNdarray([[[[ 0.30000001]]]]), 'not shown', 'not shown'] Outputs clients: [[GpuElemwise{Composite{Switch(i0, i1, (i2 * i1))},no_inplace}(GpuElemwise{Composite{Cast{float32}(GT(i0, i1))},no_inplace}.0, GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)].0, CudaNdarrayConstant{[[[[ 0.1]]]]}), GpuElemwise{Composite{Cast{float32}(GT(i0, i1))},no_inplace}(GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)].0, CudaNdarrayConstant{[[[[ 0.]]]]})]]

    HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'. HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

    Original exception: ValueError: GpuElemwise. Input dimension mis-match. Input 2 (indices start at 0) has shape[2] == 6, but the output's size on that axis is 5. Apply node that caused the error: GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)](GpuJoin.0, CudaNdarrayConstant{[[[[ 0.30000001]]]]}, GpuReshape{4}.0, GpuDimShuffle{x,0,x,x}.0) Toposort index: 1535 Inputs types: [CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, (True, True, True, True)), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, (True, False, True, True))] Inputs shapes: [(200, 192, 5, 5), (1, 1, 1, 1), (200, 192, 6, 6), (1, 192, 1, 1)] Inputs strides: [(4800, 25, 5, 1), (0, 0, 0, 0), (6912, 36, 6, 1), (0, 1, 0, 0)] Inputs values: ['not shown', CudaNdarray([[[[ 0.30000001]]]]), 'not shown', 'not shown'] Outputs clients: [[GpuElemwise{Composite{Switch(i0, i1, (i2 * i1))},no_inplace}(GpuElemwise{Composite{Cast{float32}(GT(i0, i1))},no_inplace}.0, GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)].0, CudaNdarrayConstant{[[[[ 0.1]]]]}), GpuElemwise{Composite{Cast{float32}(GT(i0, i1))},no_inplace}(GpuElemwise{Composite{((i0 + (i1 * i2)) + i3)}}[(0, 0)].0, CudaNdarrayConstant{[[[[ 0.]]]]})]]

    HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'. HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node. `

    There is a error message: ValueError: GpuElemwise. Input dimension mis-match. Input 2 (indices start at 0) has shape[2] == 6, but the output's size on that axis is 5.

    Do you have some idea about it?

    opened by tongmuyuan 3
  • Decoder is only considering unlabeled data?

    Decoder is only considering unlabeled data?

    Starting from line 205 of ladder.py it looks like only the unlabeled data is considered for the decoder. This is different from how I understood the paper, where it seem like all data is used for the encoder and decoder. If that is really not the case, shouldn't then in the fully supervised scenario the ladder turn just into a normal MLP? Where is then the boost in classification accuracy coming from?

    opened by UdoGi 3
  • Missing nargs='+'

    Missing nargs='+'

    I think that in line https://github.com/arasmus/ladder/blob/c8d26028e54c886b08d8fbccd5a6be1bbe76da52/run.py#L592 you are missing nargs='+' by chance this does not have any affect on the running of the code

    opened by udibr 3
  • Error running the code

    Error running the code

    I run: run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec sig --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full I get: File "run.py", line 649, in if train(d) is None: File "run.py", line 405, in train in_dim, data, whiten, cnorm = setup_data(p, test_set=False) File "run.py", line 240, in setup_data train_set = dataset_class("train") File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/mnist.py", line 37, in init which_sets=which_sets, **kwargs) File "/usr/local/lib/python2.7/dist-packages/fuel/datasets/hdf5.py", line 179, in init raise ValueError('which_sets should be an iterable of strings') ValueError: which_sets should be an iterable of strings

    How could I solve this? Thanks in advance.

    opened by AtrejuArtax 3
  • Unable to run CIFAR10 case

    Unable to run CIFAR10 case

    The MNIST cases are successfully replicated but I was not able to run the CIFAR10 case.

    When running the command: ./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-0 --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_baseline

    I get error as following:

    Traceback (most recent call last):
  File "run.py", line 651, in <module>
    if train(d) is None:
  File "run.py", line 405, in train
    in_dim, data, whiten, cnorm = setup_data(p, test_set=False)
  File "run.py", line 287, in setup_data
    whiten.fit(p.whiten_zca, get_data(d.train, d.train_ind))
  File "run.py", line 279, in get_data
    data = d.get_data(request=i)[d.sources.index('features')]
  File "/home/chiehchi/anaconda/lib/python2.7/site-packages/fuel/datasets/hdf5.py", line 532, in get_data
    data, shapes = self._in_memory_get_data(state, request)
  File "/home/chiehchi/anaconda/lib/python2.7/site-packages/fuel/datasets/hdf5.py", line 545, in _in_memory_get_data
    for data_source in self.data_sources]
  File "/home/chiehchi/anaconda/lib/python2.7/site-packages/fuel/utils.py", line 241, in index_within_subset
    request = self[subset_request]
  File "/home/chiehchi/anaconda/lib/python2.7/site-packages/fuel/utils.py", line 118, in __getitem__
    if key == slice(None, None, None):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

    How could I solve this? Thanks in advance!

    opened by chiehchi 2
  • Not able to run mnist_100_full

    Not able to run mnist_100_full

    Hi ! I'm newbie in this domain and i study the method proposed in the paper as a school project, I fixed the errors of the packages version, i downloaded the data but the second command of conversion doesn't work so i got the following error: Traceback (most recent call last): File "./run.py", line 652, in if train(d) is None: File "./run.py", line 406, in train in_dim, data, whiten, cnorm = setup_data(p, test_set=False) File "./run.py", line 235, in setup_data }[p.dataset] KeyError: 'a' Any one has an idea about how to solve this error ? Thank you.

    opened by Duaam 0
  • Theano Problem with old GPU back-end

    Theano Problem with old GPU back-end

    Here is my error when running the run.py file: Could you fix it?

    python3 run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full Traceback (most recent call last): File "run.py", line 14, in import theano File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/init.py", line 88, in from theano.configdefaults import config File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/configdefaults.py", line 137, in in_c_key=False) File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/configparser.py", line 287, in AddConfigVar configparam.get(root, type(root), delete_key=True) File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/configparser.py", line 335, in get self.set(cls, val_str) File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/configparser.py", line 346, in set self.val = self.filter(val) File "/home/mvidovic/virtuel_env/Ladder/lib/python3.6/site-packages/theano/configdefaults.py", line 116, in filter 'You are tring to use the old GPU back-end. ' ValueError: You are tring to use the old GPU back-end. It was removed from Theano. Use device=cuda* now. See https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29 for more information.

    opened by mcvidomi 0
  • Dimensions of mu and v for a conv layer?

    Dimensions of mu and v for a conv layer?

    If the conv layer is used and the output dimension is (m, m, c) for an input of ( k, k, c ), what is the dimensions of mu and v of Equation (1)? Is it just a c-dimensional vector or a tensor of size (m, m, c)?

    opened by sumo8291 0
  • Not able to run cifar-10

    Not able to run cifar-10

    I am able to run mnist but I get error while running cifar-10 : I get the error : TypeError: pool_2d() got an unexpected keyword argument 'ds' so I modified the line 288 in nn.py to z = pool_2d(z, ws=poolsize, stride=poolstride) and I tried running with both theano 0.8.0 / 0.8.2 and 0.9.0 but I am getting this error now:

    line 283, in pool_2d
   assert cuda.dnn.dnn_available()
AttributeError: 'module' object has no attribute 'dnn'

    nvcc --version gives the following: nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2013 NVIDIA Corporation Built on Wed_Jul_17_18:36:13_PDT_2013 Cuda compilation tools, release 5.5, V5.5.0

    Any Idea how to resolve this error ?

    opened by architkansal 0

  • "ValueError: graph contains cycles" when running "mnist_100_conv_gamma" and "cifar_4k_gamma"

    Hi, I came into some error that "ValueError: graph contains cycles" from training main-loop in theano. And this only happened when running gamma model with conv layers on both dataset "mnist" and "cifar". But if I run the "Conv-FC" model or supervised learning model on two datasets, the training can work. I am not sure how to debug for this error. Could you please help with this issue? Thanks!

    opened by liyues 0
Owner
Curious AI
Deep good. Unsupervised better.
Curious AI
GitHub
This repository contains algorithm and solver for the game WORDLE

Wordle-Cracker This repository contains algorithm and solver for the game WORDLE How to use Run the program: python main.py Type in the guess word in

YiHsin Chen 3 May 14, 2022
To solve games using AI, we will introduce the concept of a game tree followed by minimax algorithm.

To solve games using AI, we will introduce the concept of a game tree followed by minimax algorithm.

Vaibhaw 7 Oct 13, 2022
An algorithm to reach a correlated equilibrium in multiplayer games.

Correlatedpy: a python library for distributed learning of correlated equilibrium in multiplayer strategic games. View Demo · Report Bug · Request Fea

Omar Boufous 2 Feb 1, 2022
A car learns to drive in a small 2D environment using the NEAT-Algorithm with Pygame

Self-driving-car-with-Pygame A car learns to drive in a small 2D environment using the NEAT-Algorithm with Pygame Description A car has 5 sensors ("ey

Henri 3 Feb 1, 2022
Algorithm to solve Wordle correctly 100% of the time within 6 attempts.

WordleSolver © Zulkarnine, 2022. Algorithm to solve Wordle 100% of the time within 6 attempts. You can go ahead and run main.py to run it for all 2315

Zulkarnine Mahmud 69 Dec 11, 2022
TicTacToc - Simple TicTacToc game played by minimax algorithm

TicTacToc simple TicTacToc game played by minimax algorithm. This app is based o

null 5 Apr 5, 2022
A chess engine with basic AI capabilities (search for best move using MinMax algorithm)

A chess engine with basic AI capabilities (search for best move using MinMax algorithm)

Ken Wu 1 Feb 2, 2022
OS Algo Visualization - Operating system algorithm visualization using python pygame library

OS_Algo_Visualization Operating system algorithm visualization using python pyga

Krushang Satani 2 Feb 17, 2022
SelectionSortVisualization - This pygame project is helping you to understand the selection sorting algorithm

SelectionSortVisualization (If you have any comments or suggestion, please conta

Berkay IPEK 3 Feb 17, 2022
A Neural Network based chess engine and GUI made with Python and Tensorflow/Keras.

Haxaw-Chess Haxaw: Haxaw is the Neural Network based chess engine made with Python and Tensorflow/Keras. Also uses the python-chess library. (WIP: Imp

Sarthak Bharadwaj 8 Dec 10, 2022
Deep Running

Deep Running 1. Install $ pip install --user deep_running 2. Lap N Lap. Name Remarks Citation Meta 1 Mario Deeeeeep Running I was born to run. dannyso

karaage 69 Jan 31, 2022
A 2D physics sim for orbits. Made using pygame and tkinter. High degree of intractability, allowing you to create celestial bodies of a custom mass and velocity within the simulation, select what specifically is displayed, and move the camera.

Python-Orbit-Sim A 2D physics sim for orbits. Made using pygame and tkinter. High degree of intractability, allowing you to create celestial bodies of

null 3 Oct 22, 2021
A "finish the lyrics" game using Spotify, YouTube Transcript, and YouTube Search APIs, coupled with visual machine learning

Singify Introducing Singify, the party game! Challenge your friend to who knows songs better. Play random songs from your very own Spotify playlist an

Josh Wong 4 Nov 19, 2021
Jiminy, fast and portable Python/C++ simulator of poly-articulated systems with OpenAI Gym interface for reinforcement learning.

Jiminy is a fast and portable cross-platform open-source simulator for poly-articulated systems. It was built with two ideas in mind: provide a fast y

Alexis DUBURCQ 122 Dec 29, 2022
Visualizing and learning from games on chess.com

Better Your Chess What for? This project aims to help you learn from all the chess games you've played online, starting with a simple way to download

Luc d'Hauthuille 0 Apr 17, 2022
The Original Snake Game. Maneuver a snake in its burrow and earn points while avoiding the snake itself and the walls of the snake burrow.

Maneuver a snake in its burrow and earn points while avoiding the snake itself and the walls of the snake burrow. The snake grows when it eats an apple by default which can be disabled in the settings tab where you can also find all the other customization options.

null 17 Nov 12, 2022
Python Knots and Crosses game, with customizable markers and more!

Knot-and-Crosses Python Knots and Crosses game, with customizable markers and more! Features: Ability to change your marker Ability to change how many

null 4 Nov 7, 2021
Lutris helps you install and play video games from all eras and from most gaming systems.

Lutris Lutris helps you install and play video games from all eras and from most gaming systems. By leveraging and combining existing emulators, engin

Pop!_OS 2 Nov 15, 2021
Useful guides, tutorials, and FAQs related to LEGO Universe and Darkflame Universe.

Awesome Lego Universe A curated list of awesome things related to LEGO Universe. LEGO Universe was a kid-friendly massively-multiplayer online role pl

Eric Myllyoja 33 Dec 12, 2022