learning/annotated_deep_learning_paper_implementations
1
0
Fork 0
mirror of https://github.com/labmlai/annotated_deep_learning_paper_implementations.git synced 2025-10-28 20:53:41 +08:00
Code Issues Packages Projects Releases Wiki Activity

github link

Browse Source
This commit is contained in:
Varuna Jayasiri
2020-09-04 16:01:21 +05:30
parent 3e240cfe1c
commit 299cd650cd
4 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
labml_nn/transformers/__init__.py
View File

@ -1,4 +1,6 @@
"""
<a class="github-button" href="https://github.com/lab-ml/labml_nn" data-size="large" data-show-count="true" aria-label="Star lab-ml/labml_nn on GitHub">Star</a>
# Transformers
* [Multi-head attention](mha.html)

2
labml_nn/transformers/mha.py
View File

@ -1,4 +1,6 @@
"""
<a class="github-button" href="https://github.com/lab-ml/labml_nn" data-size="large" data-show-count="true" aria-label="Star lab-ml/labml_nn on GitHub">Star</a>
# Multi-Headed Attention
The implementation is inspired from [Annotated Transformer](https://nlp.seas.harvard.edu/2018/04/03/attention.html)

2
labml_nn/transformers/relative_mha.py
View File

@ -1,4 +1,6 @@
"""
<a class="github-button" href="https://github.com/lab-ml/labml_nn" data-size="large" data-show-count="true" aria-label="Star lab-ml/labml_nn on GitHub">Star</a>
# Relative Multi-head Attention
This is an implementation of