annotated_deep_learning_paper_implementations/readme.md

LabML Neural Networks

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,

The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.

We are actively maintaining this repo and adding new implementations almost weekly. Follow us on Twitter for updates.

Modules

✨ Transformers

Transformers module contains implementations for multi-headed attention and relative multi-headed attention.

• kNN-LM: Generalization through Memorization
• Feedback Transformer

✨ Recurrent Highway Networks

✨ LSTM

✨ HyperNetworks - HyperLSTM

✨ Capsule Networks

✨ Generative Adversarial Networks

• GAN with a multi-layer perceptron
• GAN with deep convolutional network
• Cycle GAN

✨ Sketch RNN

✨ Reinforcement Learning

• Proximal Policy Optimization with Generalized Advantage Estimation
• Deep Q Networks with with Dueling Network, Prioritized Replay and Double Q Network.

✨ Optimizers

• Adam
• AMSGrad
• Adam Optimizer with warmup
• Noam Optimizer
• Rectified Adam Optimizer
• AdaBelief Optimizer

Installation

pip install labml_nn

Citing LabML

If you use LabML for academic research, please cite the library using the following BibTeX entry.

@misc{labml,
 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = {LabML: A library to organize machine learning experiments},
 year = {2020},
 url = {https://lab-ml.com/},
}