learning/annotated_deep_learning_paper_implementations
1
0
Fork 0
mirror of https://github.com/labmlai/annotated_deep_learning_paper_implementations.git synced 2025-08-26 08:41:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

html

Browse Source
This commit is contained in:
Varuna Jayasiri
2022-08-26 18:06:08 +05:30
parent 1dbc2cbc04
commit c0004c9e8e
164 changed files with 3787 additions and 3747 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/scaling/zero3/finetune_neox.html
View File

@ -111,7 +111,7 @@
<a href='#section-2'>#</a>
</div>
<h4>Set the optimizers for the model</h4>
<p>Note that we pass the sharded parameters from <code class="highlight"><span></span><span class="n">get_trainable_chunk</span></code>
<p>Note that we pass the sharded parameters from <code class="highlight"><span></span><span class="n">get_trainable_chunk</span></code>
.</p>
</div>
@ -173,7 +173,7 @@
<div class='section-link'>
<a href='#section-7'>#</a>
</div>
<p>Wrap the layers with <code class="highlight"><span></span><span class="n">Zero3Layer</span></code>
<p>Wrap the layers with <code class="highlight"><span></span><span class="n">Zero3Layer</span></code>
</p>
</div>
@ -213,7 +213,7 @@
<div class='section-link'>
<a href='#section-10'>#</a>
</div>
<h4>Run the training on the node with rank <code class="highlight"><span></span><span class="n">rank</span></code>
<h4>Run the training on the node with rank <code class="highlight"><span></span><span class="n">rank</span></code>
.</h4>
</div>

58
docs/scaling/zero3/index.html
View File

File diff suppressed because one or more lines are too long