2 Repositories
Python adamw Libraries
AdamW optimizer for bfloat16 models in pytorch.
Image source AdamW optimizer for bfloat16 models in pytorch. Bfloat16 is currently an optimal tradeoff between range and relative error for deep netwo
8 Nov 20, 2022
AdamW optimizer and cosine learning rate annealing with restarts
AdamW optimizer and cosine learning rate annealing with restarts This repository contains an implementation of AdamW optimization algorithm and cosine
133 Dec 20, 2022