Initializer

Python API

Popular initialization methods for parameter values (Tensor objects).

Example usages:

from singa import tensor
from singa import initializer

x = tensor.Tensor((3, 5))
initializer.uniform(x, 3, 5) # use both fan_in and fan_out
initializer.uniform(x, 3, 0)  # use only fan_in
singa.initializer.uniform(t, fan_in=0, fan_out=0)

Initialize the values of the input tensor following a uniform distribution with specific bounds.

Parameters
  • fan_in (int) – for the weight Tensor of a convolution layer, fan_in = nb_channel * kh * kw; for dense layer, fan_in = input_feature_length

  • fan_out (int) – for the convolution layer weight Tensor, fan_out = nb_filter * kh * kw; for the weight Tensor of a dense layer, fan_out = output_feature_length

Ref: [Bengio and Glorot 2010]: Understanding the difficulty of training deep feedforward neuralnetworks.

singa.initializer.gaussian(t, fan_in=0, fan_out=0)

Initialize the values of the input tensor following a Gaussian distribution with specific std.

Parameters
  • fan_in (int) – for the weight Tensor of a convolution layer, fan_in = nb_channel * kh * kw; for dense layer, fan_in = input_feature_length

  • fan_out (int) – for the convolution layer weight Tensor, fan_out = nb_filter * kh * kw; for the weight Tensor of a dense layer, fan_out = output_feature_length

Ref Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

CPP API