learning/annotated_deep_learning_paper_implementations
1
0
Fork 0
mirror of https://github.com/labmlai/annotated_deep_learning_paper_implementations.git synced 2025-08-15 10:11:39 +08:00
Code Issues Packages Projects Releases Wiki Activity

📊 tracker debug attention

Browse Source
This commit is contained in:
Varuna Jayasiri
2020-09-03 09:32:02 +05:30
parent 4e15860907
commit 22215f8d07
3 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
labml_nn/transformers/utils.py Normal file
View File

@ -0,0 +1,6 @@
import torch
def subsequent_mask(seq_len):
mask = torch.tril(torch.ones(seq_len, seq_len)).to(torch.bool).unsqueeze(-1)
return mask