Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

Giora Simchoni

Last update: Nov 2, 2022

Related tags

Deep Learning lmmnn

Overview

LMMNN

Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

This is the working directory for our Neurips 2021 submission.

For full details see the paper and lmmnn_neurips2021_additional_material.pdf.

For running the simulations use the simulate.py file, like so:

python simulate.py --conf conf.yaml --out res.csv

The --conf attribute accepts a yaml file such as conf.yaml which you can change.

To run various real data experiments see the jupyter notebooks in the notebooks folder. We cannot unfortunately attach the actual datasets, see paper for details.

For using LMMNN with your own data use the NLL loss layer as shown in notebooks and simulation.

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

2 Nov 14, 2022

This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effects in Video."

Omnimatte in PyTorch This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effect

728 Dec 28, 2022

An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

ImageCompressionSimulation An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects o

1 Dec 11, 2021

improvement of CLIP features over the traditional resnet features on the visual question answering, image captioning, navigation and visual entailment tasks.

CLIP-ViL In our paper "How Much Can CLIP Benefit Vision-and-Language Tasks?", we show the improvement of CLIP features over the traditional resnet fea

310 Dec 28, 2022

Cancer-and-Tumor-Detection-Using-Inception-model - In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks, specifically here the Inception model by google.

Cancer-and-Tumor-Detection-Using-Inception-model In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks

1 Jan 1, 2022

Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

Complex-Valued Neural Networks (CVNN) Done by @NEGU93 - J. Agustin Barrachina Using this library, the only difference with a Tensorflow code is that y

1 Nov 12, 2021

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data Au

14 Nov 28, 2022

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

210 Jan 4, 2023

DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

214 Jan 8, 2023

Comments

ValueError: dimension mismatch
Dear Giora,

Stunning work on combining neural networks with mixed effects models! I'm interested in applying this to a binary classification task.

Since I couldn't find an example that uses mode 'glmm' among your notebooks, I worked through the imdb.ipynb notebook with the following adjustments added throughout the script.

# make y binary imdb = imdb.assign( score = lambda dataframe: dataframe['score'].map(lambda score: 1 if score >= 7 else 0) )

# specify mode mode = 'glmm'

# Model adjustments, though this is not critical here y_pred_output = Dense(1, activation = 'sigmoid')(out_hidden) optimizer = keras.optimizers.Adam(learning_rate=0.001) model.compile(optimizer= optimizer)

Running the notebook, I receive the following error:

ValueError: dimension mismatch

I think the problem stems from the calculation of 'b_hat'. In 'calc_b_hat.py', lines 99 to 129, 'b_hat' seems to be calculated only for 'z0', not for 'z1' as well.

Then in 'nn.py', lines 532 to 533 produce the error message:

y_pred = model.predict([X_test[x_cols], dummy_y_test] + X_test_z_cols).reshape( X_test.shape[0]) + Z_test @ b_hat

Since qs > 1 (2 in this case), Z_test combines the levels/categories of both z0 and z1. However, the length of b_hat is qs[0], i.e., the levels/categories in z0. The length of b_hat should be qs[0]+qs[1], no? Hence the dimension mismatch.

Should lines 99 to 129 in 'calc_b_hat' be adjusted to include an outermost loop through range(qs), then in the end stacking the b_hats of both variables?

Appreciate any help on this! Do you happen to have a working example with mode 'glmm'?
enhancement
opened by saschagobel 3

Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

Related tags

Overview

LMMNN

Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

You might also like...

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effects in Video."

An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

improvement of CLIP features over the traditional resnet features on the visual question answering, image captioning, navigation and visual entailment tasks.

Cancer-and-Tumor-Detection-Using-Inception-model - In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks, specifically here the Inception model by google.

Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

Comments

ValueError: dimension mismatch

Owner

Giora Simchoni

Categorical Depth Distribution Network for Monocular 3D Object Detection

A working implementation of the Categorical DQN (Distributional RL).

Code for "On the Effects of Batch and Weight Normalization in Generative Adversarial Networks"

Jetson Nano-based smart camera system that measures crowd face mask usage in real-time.

Measures input lag without dedicated hardware, performing motion detection on recorded or live video

Group project for MFIN7036. Our goal is to predict firm profitability with text-based competition measures.

Random-Afg - Afghanistan Random Old Idz Cloner Tools

DeepHawkeye is a library to detect unusual patterns in images using features from pretrained neural networks

Static Features Classifier - A static features classifier for Point-Could clusters using an Attention-RNN model

Random Walk Graph Neural Networks