Loss¶
Loss module includes a set of training loss implmentations. Some are converted from C++ implementation, and the rest are implemented directly using python Tensor.
Example usage:
from singa import tensor
from singa import loss
x = tensor.Tensor((3, 5))
x.uniform(0, 1) # randomly genearte the prediction activation
y = tensor.from_numpy(np.array([0, 1, 3], dtype=np.int)) # set the truth
f = loss.SoftmaxCrossEntropy()
l = f.forward(True, x, y) # l is tensor with 3 loss values
g = f.backward() # g is a tensor containing all gradients of x w.r.t l
-
class
singa.loss.
Loss
¶ Bases:
object
Base loss class.
Subclasses that wrap the C++ loss classes can use the inherited foward, backward, and evaluate functions of this base class. Other subclasses need to override these functions
-
backward
()¶ Returns: the grad of x w.r.t. the loss
-
evaluate
(flag, x, y)¶ Parameters: Returns: the averaged loss for all samples in x.
-
forward
(flag, x, y)¶ Compute the loss values.
Parameters: Returns: a tensor of floats for the loss values, one per sample
-
-
class
singa.loss.
SoftmaxCrossEntropy
¶ Bases:
singa.loss.Loss
This loss function is a combination of SoftMax and Cross-Entropy loss.
It converts the inputs via SoftMax function and then computes the cross-entropy loss against the ground truth values.
For each sample, the ground truth could be a integer as the label index; or a binary array, indicating the label distribution. The ground truth tensor thus could be a 1d or 2d tensor. The data/feature tensor could 1d (for a single sample) or 2d for a batch of samples.
-
class
singa.loss.
SquaredError
¶ Bases:
singa.loss.Loss
This loss evaluates the squared error between the prediction and the truth values.
It is implemented using Python Tensor operations.
-
backward
()¶ Compute the gradient of x w.r.t the error.
Returns: x - y
-
evaluate
(flag, x, y)¶ Compuate the averaged error.
Returns: a float value as the averaged error
-
forward
(flag, x, y)¶ Compute the error as 0.5 * ||x-y||^2.
Parameters: Returns: a Tensor with one error value per sample
-