learning/annotated_deep_learning_paper_implementations
1
0
Fork 0
mirror of https://github.com/labmlai/annotated_deep_learning_paper_implementations.git synced 2025-08-06 15:22:21 +08:00
Code Issues Packages Projects Releases Wiki Activity

rope links

Browse Source
This commit is contained in:
Varuna Jayasiri
2022-02-23 15:10:49 +05:30
parent cd03271fb6
commit d747e463a4
3 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
labml_nn/__init__.py
View File

@ -22,6 +22,7 @@ implementations.
* [Transformer building blocks](transformers/models.html) * [Transformer building blocks](transformers/models.html)
* [Transformer XL](transformers/xl/index.html) * [Transformer XL](transformers/xl/index.html)
* [Relative multi-headed attention](transformers/xl/relative_mha.html) * [Relative multi-headed attention](transformers/xl/relative_mha.html)
* [Rotary Positional Embeddings](transformers/rope/index.html)
* [Compressive Transformer](transformers/compressive/index.html) * [Compressive Transformer](transformers/compressive/index.html)
* [GPT Architecture](transformers/gpt/index.html) * [GPT Architecture](transformers/gpt/index.html)
* [GLU Variants](transformers/glu_variants/simple.html) * [GLU Variants](transformers/glu_variants/simple.html)

3
labml_nn/transformers/__init__.py
View File

@ -22,6 +22,9 @@ and derivatives and enhancements of it.
This implements Transformer XL model using This implements Transformer XL model using
[relative multi-head attention](xl/relative_mha.html) [relative multi-head attention](xl/relative_mha.html)
## [Rotary Positional Embeddings](rope/index.html)
This implements Rotary Positional Embeddings (RoPE)
## [Compressive Transformer](compressive/index.html) ## [Compressive Transformer](compressive/index.html)
This is an implementation of compressive transformer This is an implementation of compressive transformer

1
readme.md
View File

@ -24,6 +24,7 @@ implementations almost weekly.
* [Transformer building blocks](https://nn.labml.ai/transformers/models.html) * [Transformer building blocks](https://nn.labml.ai/transformers/models.html)
* [Transformer XL](https://nn.labml.ai/transformers/xl/index.html) * [Transformer XL](https://nn.labml.ai/transformers/xl/index.html)
* [Relative multi-headed attention](https://nn.labml.ai/transformers/xl/relative_mha.html) * [Relative multi-headed attention](https://nn.labml.ai/transformers/xl/relative_mha.html)
* [Rotary Positional Embeddings](https://nn.labml.ai/transformers/rope/index.html)
* [Compressive Transformer](https://nn.labml.ai/transformers/compressive/index.html) * [Compressive Transformer](https://nn.labml.ai/transformers/compressive/index.html)
* [GPT Architecture](https://nn.labml.ai/transformers/gpt/index.html) * [GPT Architecture](https://nn.labml.ai/transformers/gpt/index.html)
* [GLU Variants](https://nn.labml.ai/transformers/glu_variants/simple.html) * [GLU Variants](https://nn.labml.ai/transformers/glu_variants/simple.html)