The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Samuel Lippl

Last update: Feb 5, 2022

Related tags

Deep Learning implicit-bias-glns

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

This folder contains the code to reproduce the data in "The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks."

To reproduce the 'data' folder:

Activate the environment provided as the 'environment.yml' file.
Run 'shallow_gln.sh', 'shallow_gln_cp.sh', 'shallow_gln_cp_2.sh', 'deep_gln.sh', 'deep_gln_cp.sh', 'relu_net.sh', and 'relu_net_cp.sh'. The convex optimization scripts (end in cp) must be run after their gradient descent counterparts (do not end in cp).

We then created an R package to package the cleaned datasets ('glnsanalysis', available in the folder 'implicit-bias-glns-analysis'). To access the datasets, build the packages. The notebooks in the folder 'vignettes' reproduce the figures.

Python script for Linear, Non-Linear Convection, Burger’s & Poisson Equation in 1D & 2D, 1D Diffusion Equation using Standard Wall Function, 2D Heat Conduction Convection equation with Dirichlet & Neumann BC, full Navier-Stokes Equation coupled with Poisson equation for Cavity and Channel flow in 2D using Finite Difference Method & Finite Volume Method.

Navier-Stokes-numerical-solution-using-Python- Python script for Linear, Non-Linear Convection, Burger’s & Poisson Equation in 1D & 2D, 1D D

89 Jan 4, 2023

Simple Linear 2nd ODE Solver GUI - A 2nd constant coefficient linear ODE solver with simple GUI using euler's method

Simple_Linear_2nd_ODE_Solver_GUI Description It is a 2nd constant coefficient li

4 Feb 5, 2022

Hitters Linear Regression - Hitters Linear Regression With Python

Hitters_Linear_Regression Kullanacağımız veri seti Carnegie Mellon Üniversitesi'

2 Jan 26, 2022

Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Medical-Transformer Pytorch Code for the paper "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation" About this repo: This repo

615 Dec 25, 2022

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

4 May 20, 2022

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

2 Nov 14, 2022

Learning hidden low dimensional dyanmics using a Generalized Onsager Principle and neural networks

OnsagerNet Learning hidden low dimensional dyanmics using a Generalized Onsager Principle and neural networks This is the original pyTorch implemenati

3 Aug 24, 2022

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias

Counterfactual VQA (CF-VQA) This repository is the Pytorch implementation of our paper "Counterfactual VQA: A Cause-Effect Look at Language Bias" in C

94 Dec 3, 2022

A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK

Pytorch-MBNet A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK Training To train a new model, please ru

46 Dec 28, 2022

Owner

Samuel Lippl

Neuroscience, statistics, mathematics

GitHub

Linear algebra python - Number of operations and problems in Linear Algebra and Numerical Linear Algebra

Linear algebra in python Number of operations and problems in Linear Algebra and

5 Oct 9, 2022

Official Pytorch implementation of "Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021)

Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021) Official Pytorch implementation of Unbiased Classification

17 Jan 1, 2023

Training vision models with full-batch gradient descent and regularization

Stochastic Training is Not Necessary for Generalization -- Training competitive vision models without stochasticity This repository implements trainin

32 Jan 6, 2023

Plotting points that lie on the intersection of the given curves using gradient descent.

Plotting intersection of curves using gradient descent Webapp Link ---> What's the app about Why this app Plotting functions and their intersection. A

2 Jan 9, 2022

Fibonacci Method Gradient Descent

An implementation of the Fibonacci method for gradient descent, featuring a TKinter GUI for inputting the function / parameters to be examined and a matplotlib plot of the function and results.

1 Jan 28, 2022

With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function

With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function. At the moment, only TensorFlow sequential models are supported. Interfaces to either the Pyomo or Gurobi modeling environments are offered.

40 Dec 27, 2022

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Related tags

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

You might also like...

Simple Linear 2nd ODE Solver GUI - A 2nd constant coefficient linear ODE solver with simple GUI using euler's method

Hitters Linear Regression - Hitters Linear Regression With Python

Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

Learning hidden low dimensional dyanmics using a Generalized Onsager Principle and neural networks

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias

A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK

Owner

Samuel Lippl

Linear algebra python - Number of operations and problems in Linear Algebra and Numerical Linear Algebra

Official Pytorch implementation of "Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021)

Training vision models with full-batch gradient descent and regularization

Plotting points that lie on the intersection of the given curves using gradient descent.

Fibonacci Method Gradient Descent

With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function

Official implementation of Generalized Data Weighting via Class-level Gradient Manipulation (NeurIPS 2021).

A PyTorch Implementation of Gated Graph Sequence Neural Networks (GGNN)

A PyTorch Implementation of Gated Graph Sequence Neural Networks (GGNN)

Proximal Backpropagation - a neural network training algorithm that takes implicit instead of explicit gradient steps