Loss¶
Loss module includes a set of training loss implmentations. Some are converted from C++ implementation, and the rest are implemented directly using python Tensor.
Example usage:
from singa import tensor
from singa import loss
x = tensor.Tensor((3, 5))
x.uniform(0, 1) # randomly genearte the prediction activation
y = tensor.from_numpy(np.array([0, 1, 3], dtype=np.int)) # set the truth
f = loss.SoftmaxCrossEntropy()
l = f.forward(True, x, y) # l is tensor with 3 loss values
g = f.backward() # g is a tensor containing all gradients of x w.r.t l
-
class
singa.loss.Loss¶ Bases:
objectBase loss class.
Subclasses that wrap the C++ loss classes can use the inherited foward, backward, and evaluate functions of this base class. Other subclasses need to override these functions
-
backward()¶ - Returns
the grad of x w.r.t. the loss
-
evaluate(flag, x, y)¶
-
forward(flag, x, y)¶ Compute the loss values.
-
-
class
singa.loss.SigmoidCrossEntropy(epsilon=1e-08)¶ Bases:
singa.loss.LossThis loss evaluates the cross-entropy loss between the prediction and the truth values with the prediction probability generated from Sigmoid.
-
backward()¶ Compute the gradient of loss w.r.t to x.
- Returns
dx = pi - yi.
-
evaluate(flag, x, y)¶ Compuate the averaged error.
- Returns
a float value as the averaged error
-
-
class
singa.loss.SoftmaxCrossEntropy¶ Bases:
singa.loss.LossThis loss function is a combination of SoftMax and Cross-Entropy loss.
It converts the inputs via SoftMax function and then computes the cross-entropy loss against the ground truth values.
For each sample, the ground truth could be a integer as the label index; or a binary array, indicating the label distribution. The ground truth tensor thus could be a 1d or 2d tensor. The data/feature tensor could 1d (for a single sample) or 2d for a batch of samples.
-
class
singa.loss.SquaredError¶ Bases:
singa.loss.LossThis loss evaluates the squared error between the prediction and the truth values.
It is implemented using Python Tensor operations.
-
backward()¶ Compute the gradient of x w.r.t the error.
- Returns
x - y
-
evaluate(flag, x, y)¶ Compuate the averaged error.
- Returns
a float value as the averaged error
-
forward(flag, x, y)¶ Compute the error as 0.5 * ||x-y||^2.
-
