diff --git a/labml_nn/__init__.py b/labml_nn/__init__.py index 09da56dd..9d797d76 100644 --- a/labml_nn/__init__.py +++ b/labml_nn/__init__.py @@ -50,7 +50,10 @@ and #### ✨ [Optimizers](https://lab-ml.com/labml_nn/optimizers/) * [Adam](https://lab-ml.com/labml_nn/optimizers/adam.html) * [AMSGrad](https://lab-ml.com/labml_nn/optimizers/amsgrad.html) +* [Adam Optimizer with warmup](https://lab-ml.com/labml_nn/optimizers/adam_warmup.html) +* [Noam Optimizer](https://lab-ml.com/labml_nn/optimizers/noam.html) * [Rectified Adam Optimizer](https://lab-ml.com/labml_nn/optimizers/radam.html) +* [AdaBelief Optimizer](https://lab-ml.com/labml_nn/optimizers/adabelief.html) ### Installation diff --git a/labml_nn/optimizers/__init__.py b/labml_nn/optimizers/__init__.py index 47be448c..8a1a77f5 100644 --- a/labml_nn/optimizers/__init__.py +++ b/labml_nn/optimizers/__init__.py @@ -11,7 +11,10 @@ summary: > ## Optimizer Implementations * [Adam Optimizer](adam.html) * [AMSGrad Optimizer](amsgrad.html) +* [Adam Optimizer with warmup](adam_warmup.html) +* [Noam Optimizer](noam.html) * [Rectified Adam Optimizer](radam.html) +* [AdaBelief Optimizer](adabelief.html) This [MNIST example](mnist_experiment.html) uses these optimizers. diff --git a/readme.md b/readme.md index 68bb1807..b21a9da2 100644 --- a/readme.md +++ b/readme.md @@ -49,7 +49,10 @@ and #### ✨ [Optimizers](https://lab-ml.com/labml_nn/optimizers/) * [Adam](https://lab-ml.com/labml_nn/optimizers/adam.html) * [AMSGrad](https://lab-ml.com/labml_nn/optimizers/amsgrad.html) +* [Adam Optimizer with warmup](https://lab-ml.com/labml_nn/optimizers/adam_warmup.html) +* [Noam Optimizer](https://lab-ml.com/labml_nn/optimizers/noam.html) * [Rectified Adam Optimizer](https://lab-ml.com/labml_nn/optimizers/radam.html) +* [AdaBelief Optimizer](https://lab-ml.com/labml_nn/optimizers/adabelief.html) ### Installation