learning/annotated_deep_learning_paper_implementations
1
0
Fork 0
mirror of https://github.com/labmlai/annotated_deep_learning_paper_implementations.git synced 2025-08-26 08:41:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

use forward

Browse Source
This commit is contained in:
Varuna Jayasiri
2021-02-02 10:22:22 +05:30
parent c5d47067be
commit d3790d708b
26 changed files with 188 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/transformers/mha.html
View File

@ -155,7 +155,7 @@ This is used to transform <strong>key</strong>, <strong>query</strong>, and <str
</div>
<div class='code'>
<div class="highlight"><pre><span class="lineno">45</span> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">Tensor</span><span class="p">):</span></pre></div>
<div class="highlight"><pre><span class="lineno">45</span> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">Tensor</span><span class="p">):</span></pre></div>
</div>
</div>
<div class='section' id='section-7'>
@ -377,7 +377,7 @@ They have shape <code>[seq_len, batch_size, d_model]</code>.</p>
query at position <code>i</code> has access to key-value at position <code>j</code>.</p>
</div>
<div class='code'>
<div class="highlight"><pre><span class="lineno">121</span> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span>
<div class="highlight"><pre><span class="lineno">121</span> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span>
<span class="lineno">122</span> <span class="n">query</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">Tensor</span><span class="p">,</span>
<span class="lineno">123</span> <span class="n">key</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">Tensor</span><span class="p">,</span>
<span class="lineno">124</span> <span class="n">value</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">Tensor</span><span class="p">,</span>