Implementation of linesearch Optimization Algorithms in Python

Paul

Last update: Dec 6, 2022

Related tags

Machine Learning nonlinear-optimization conjugate-gradient-descent gradient-descent-algorithm newtons-method trust-region-dogleg-algorithm armijo-backtrack rosenbrock-function-optimization bfgs-algorithm

Overview

Nonlinear Optimization Algorithms

During my time as Scientific Assistant at the Karlsruhe Institute of Technology (Germany) I implemented various Optimization Algorithms solving unrestricted nonlinear Problems; Gradient-Descent-Method, Newton-Method, Conjugate-Gradient-Descent-Method, BFGS-Method and a Trust-Region-Method in Python.

In addition, I implemented an Armijo linesearch.

The code is implemented in an object-oriented manner, whereby each method is implemented in a class (bfgs.py, cg.py, gradv.py, newtonm.py and tr.py) and executed via the ros_test.py script. In the script ros_test.py the Rosenbrock function was implemented, which is minimized to a given starting point x_0 with each method.

You might also like...

Bonsai: Gradient Boosted Trees + Bayesian Optimization

Bonsai is a wrapper for the XGBoost and Catboost model training pipelines that leverages Bayesian optimization for computationally efficient hyperparameter tuning.

24 Oct 27, 2022

Bayesian optimization based on Gaussian processes (BO-GP) for CFD simulations.

BO-GP Bayesian optimization based on Gaussian processes (BO-GP) for CFD simulations. The BO-GP codes are developed using GPy and GPyOpt. The optimizer

8 Mar 31, 2022

Tools for mathematical optimization region

15 Nov 30, 2022

Metric learning algorithms in Python

metric-learn: Metric Learning in Python metric-learn contains efficient Python implementations of several popular supervised and weakly-supervised met

1.3k Dec 28, 2022

Python-based implementations of algorithms for learning on imbalanced data.

ND DIAL: Imbalanced Algorithms Minimalist Python-based implementations of algorithms for imbalanced learning. Includes deep and representational learn

220 Dec 13, 2022

Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

4.9k Jan 5, 2023

A Python-based application demonstrating various search algorithms, namely Depth-First Search (DFS), Breadth-First Search (BFS), and A* Search (Manhattan Distance Heuristic)

A Python-based application demonstrating various search algorithms, namely Depth-First Search (DFS), Breadth-First Search (BFS), and the A* Search (using the Manhattan Distance Heuristic)

17 Aug 14, 2022

scikit-multimodallearn is a Python package implementing algorithms multimodal data.

scikit-multimodallearn is a Python package implementing algorithms multimodal data. It is compatible with scikit-learn, a popul

12 Jun 29, 2022

Uplift modeling and causal inference with machine learning algorithms

Disclaimer This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to chang

3.7k Jan 7, 2023

Owner

Paul

M.Sc student at Columbia University and Karlsruhe Institute of Technology. Passionate about Mathematical Modeling, Statistics and Optimization.

GitHub

This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch

This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch. It uses a simple TestEnvironment to test the algorithm

59 Dec 9, 2022

Implementation of different ML Algorithms from scratch, written in Python 3.x

393 Nov 29, 2022

Machine learning algorithms implementation

Machine learning algorithms implementation This repository consisits of implementation of various machine learning algorithms. The algorithms implemen

1 Jan 3, 2022

Pyomo is an object-oriented algebraic modeling language in Python for structured optimization problems.

Pyomo is a Python-based open-source software package that supports a diverse set of optimization capabilities for formulating and analyzing optimization models. Pyomo can be used to define symbolic problems, create concrete problem instances, and solve these instances with standard solvers.

1.4k Dec 28, 2022

A Python step-by-step primer for Machine Learning and Optimization

early-ML Presentation General Machine Learning tutorials A Python step-by-step primer for Machine Learning and Optimization This github repository gat

8 Dec 1, 2022

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective. 10x Larger Models 10x Faster Trainin

8.4k Dec 30, 2022

CS 7301: Spring 2021 Course on Advanced Topics in Optimization in Machine Learning

141 Nov 10, 2022

Bayesian optimization in JAX

26 May 11, 2022

jaxfg - Factor graph-based nonlinear optimization library for JAX.

Factor graphs + nonlinear optimization in JAX

134 Dec 21, 2022

MooGBT is a library for Multi-objective optimization in Gradient Boosted Trees.

MooGBT is a library for Multi-objective optimization in Gradient Boosted Trees. MooGBT optimizes for multiple objectives by defining constraints on sub-objective(s) along with a primary objective. The constraints are defined as upper bounds on sub-objective loss function. MooGBT uses a Augmented Lagrangian(AL) based constrained optimization framework with Gradient Boosted Trees, to optimize for multiple objectives.

66 Dec 6, 2022