This commit is contained in:
Varuna Jayasiri
2022-08-26 18:06:08 +05:30
parent 1dbc2cbc04
commit c0004c9e8e
164 changed files with 3787 additions and 3747 deletions

View File

@ -125,7 +125,7 @@
</div>
<h3>Basic english tokenizer</h3>
<p>We use character level tokenizer in this experiment. You can switch by setting,</p>
<pre class="highlight lang-"><code><span></span><span class="s1">&#39;tokenizer&#39;</span><span class="p">:</span> <span class="s1">&#39;basic_english&#39;</span></code></pre>
<pre class="highlight lang-"><code><span></span><span class="s1">&#39;tokenizer&#39;</span><span class="p">:</span> <span class="s1">&#39;basic_english&#39;</span></code></pre>
<p>in the configurations dictionary when starting the experiment.</p>
</div>