cf565bcc1d
cleanup
2024-06-18 11:09:02 +05:30
418d1ec44a
RWKV docs
2024-03-17 17:47:39 +05:30
df9e1af615
RWKV docs
2024-03-17 17:45:08 +05:30
7db6e92376
RWKV ( #222 )
...
* rwkv-init
* annotations
* Re-added docs
* make dir if not exist
* Add RWKV paper and update doc index
* add train loop
* experiment
---------
Co-authored-by: Jacob Hatef <hatef.4@buckeyemail.buckeyemail.osu.edu>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
2024-03-17 17:36:15 +05:30
5ec0f70855
Fix formula typo in Relative MHA ( #242 )
...
${(\textcolor{lightgreen}{\mathbf{A + C}})}_{i,j} = Q_i^\top K_j + \textcolor{orange}{v^\top} K_j$
2024-03-02 14:19:06 +05:30
fea91b9699
Cleanup group norm Cifar experiment ( #240 )
...
This experiment is not written in the previous format in group normalization and appears longer. With modifications, this code will normalize and be shorter
2024-03-02 14:17:39 +05:30
84ad3f9783
Update unet.py ( #239 ) fix typo
2024-03-02 14:13:34 +05:30
a0679ecd90
title
2024-01-12 13:21:54 +05:30
083988f411
Merge pull request #232 from Etienne248/patch-1
...
Fix a typo in the formula of RoPE
2024-01-12 13:17:19 +05:30
4135eda943
title
2024-01-12 13:15:04 +05:30
bc5565b84c
Fix a typo in the formula of RoPE
2023-12-08 15:50:21 +01:00
36a374ed76
Merge pull request #226 from MrYxJ/patch-1
...
Fix a typo in the formula of ALiBi.
2023-11-17 17:42:17 +00:00
830161b299
Update __init__.py
...
This formula is wrong, there is one symbol '-' missing in front of the 1, which will affect people's understanding when reading. What is expressed here is that the position of the ith token is increasing from -(i-1) to 0, so it should be -1.
2023-11-14 00:30:26 +08:00
4d922e838f
Add backticks to mask'shape
2023-11-10 19:51:37 +08:00
f42c0e9cf4
right shift example comment fix
2023-11-07 09:28:22 +00:00
a2d6e805a6
fix beta expression in ddpm docs
2023-11-07 09:13:19 +00:00
334fb05fb2
Merge pull request #221 from lizhuoq/fix
...
fix: fix cls_token bug in vit.
2023-11-07 09:06:12 +00:00
3a759f45b0
minor typo: instance norm -> group norm
2023-11-06 21:50:35 +08:00
6576e98498
minor typo in the type hint of ddpm/unet
2023-11-06 21:48:39 +08:00
ffafaf1df7
fix: fix cls_token bug in vit.
2023-11-06 11:28:45 +08:00
9a42ac2697
arxiv.org links
2023-10-24 14:42:32 +01:00
732aedcfc6
Minor typo DDPM->DDIM
2023-09-21 18:15:15 +02:00
51955fdf89
Update __init__.py
...
Corrected typo in latex code
2023-08-15 20:59:13 +03:00
ec3c7366b7
labml update
2023-08-10 19:37:55 +05:30
b01848a2a0
Merge pull request #196 from YangWu1227/fix-comment-typo
...
fix: potential typo in resnet comments
2023-07-15 08:35:09 +05:30
b43fb807a8
sophia speed up
2023-07-15 08:30:41 +05:30
0101618de6
links
2023-07-14 21:27:44 +05:30
8db330dd22
sophia-g docs
2023-07-14 21:25:08 +05:30
7c02294e7c
sophia exp
2023-07-14 16:44:45 +05:30
f45ca5ee69
sophia wip
2023-07-14 15:40:26 +05:30
1e0007796d
fix: potential typo in resnet comments
2023-07-10 02:34:22 -05:00
689842ae09
Merge pull request #193 from hlyang1992/master
...
Add the missing negative sign in the formula.
2023-06-30 11:38:26 +01:00
1b51fbc5d0
Merge pull request #192 from andreemic/patch-1
...
Update unet.py: fix typo
2023-06-30 11:38:13 +01:00
9c288586bc
Merge pull request #182 from csl122/patch-1
...
add two missing param comments for UNetModel class
2023-06-30 11:37:56 +01:00
ce0cdb676f
ja translation
2023-06-30 16:03:07 +05:30
2eccd8bec6
Add the missing negative sign in the formula.
2023-06-28 06:52:45 +08:00
7cc3a7aa27
Update unet.py: fix typo
...
changed "suck" -> "such"
2023-06-21 17:11:10 +02:00
084fc89d63
add two missing param comments for UNetModel class
...
comments for tf_layers and d_cond were missing, added.
2023-05-08 16:50:28 +01:00
d4c3335525
link to translations
2023-04-02 14:03:23 +05:30
c5685c9ffe
remove app.labml.ai links
2023-04-02 12:10:18 +05:30
97e53c0f3d
fix glu variants links
2023-04-02 12:00:23 +05:30
c0b4a37870
remove sponsors
2023-02-24 18:40:20 +05:30
d198a44fa2
fix dropout ddpm.unet
2023-02-17 14:28:24 +05:30
8053f3f408
Change default dropout rate.
2023-02-16 04:28:35 +08:00
9e807c8164
Add activation for time embedding and dropout
2023-02-16 04:24:31 +08:00
ce816b9be3
delete unnecessary code and replace
2023-01-19 23:40:23 +09:00
59dde18a94
delete torch.nn.functional
2023-01-16 22:51:53 +09:00
b1f5c8e3a5
typo notes
2022-12-27 08:27:44 +00:00
963ed5575d
Merge pull request #158 from amqdn/dev
...
Fix typo in SD
2022-12-24 12:56:52 +00:00
3ec5fa9f3d
fix typo mha
2022-12-24 12:53:36 +00:00
