Initializer¶
Python API¶
Popular initialization methods for parameter values (Tensor objects).
Example usages:
from singa import tensor
from singa import initializer
x = tensor.Tensor((3, 5))
initializer.uniform(x, 3, 5) # use both fan_in and fan_out
initializer.uniform(x, 3, 0) # use only fan_in
-
singa.initializer.uniform(t, fan_in=0, fan_out=0)¶ Initialize the values of the input tensor following a uniform distribution with specific bounds.
Parameters: - fan_in (int) – for the weight Tensor of a convolution layer, fan_in = nb_channel * kh * kw; for dense layer, fan_in = input_feature_length
- fan_out (int) – for the convolution layer weight Tensor, fan_out = nb_filter * kh * kw; for the weight Tensor of a dense layer, fan_out = output_feature_length
Ref: [Bengio and Glorot 2010]: Understanding the difficulty of training deep feedforward neuralnetworks.
-
singa.initializer.gaussian(t, fan_in=0, fan_out=0)¶ Initialize the values of the input tensor following a Gaussian distribution with specific std.
Parameters: - fan_in (int) – for the weight Tensor of a convolution layer, fan_in = nb_channel * kh * kw; for dense layer, fan_in = input_feature_length
- fan_out (int) – for the convolution layer weight Tensor, fan_out = nb_filter * kh * kw; for the weight Tensor of a dense layer, fan_out = output_feature_length
Ref Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
