diff --git a/docs/normalization/weight_standardization/readme.html b/docs/normalization/weight_standardization/readme.html new file mode 100644 index 00000000..c0466887 --- /dev/null +++ b/docs/normalization/weight_standardization/readme.html @@ -0,0 +1,118 @@ + + + + + + + + + + + + + + + + + + + + + + + Weight Standardization + + + + + + + + +
+
+
+ +
+
+ +
+ +
+
+
+ + + + + + \ No newline at end of file diff --git a/labml_nn/normalization/weight_standardization/readme.md b/labml_nn/normalization/weight_standardization/readme.md new file mode 100644 index 00000000..5b8fe70a --- /dev/null +++ b/labml_nn/normalization/weight_standardization/readme.md @@ -0,0 +1,6 @@ +# [Weight Standardization](https://nn.labml.ai/normalization/weight_standardization/index.html) + +This is a [PyTorch](https://pytorch.org) implementation of Weight Standardization from the paper + [Micro-Batch Training with Batch-Channel Normalization and Weight Standardization](https://arxiv.org/abs/1903.10520). +We also have an +[annotated implementation of Batch-Channel Normalization](https://nn.labml.ai/normalization/batch_channel_normalization/index.html).