The feed-forward neural net. More...

#include <feed_forward_net.h>

Collaboration diagram for singa::FeedForwardNet:

Public Member Functions
	~FeedForwardNet ()
	Delete all layers.

std::shared_ptr< Layer >	Add (std::shared_ptr< Layer > layer)
	Add a layer with the assumption that. More...

std::shared_ptr< Layer >	Add (const LayerConf &conf, const Shape *sample_shape=nullptr)
	Add a layer by providing its configuration, and setup it. More...

void	Compile (bool shuffle, Optimizer opt, Loss loss, Metric *metric)
	Set some fields used for training and evaluating the neural net. More...

void	Compile (bool shuffle, bool to_register, std::shared_ptr< Updater > updater, Loss loss, Metric metric)
	Set some fields used for training and evaluating the neural net. More...

void	Train (size_t batchsize, int nb_epoch, const Tensor &x, const Tensor &y, float val_split=0.0f)
	Conduct the training giving the training data 'x' and label 'y'. More...

void	Train (size_t batchsize, int nb_epoch, const Tensor &x, const Tensor &y, const Tensor &val_x, const Tensor &val_y)
	Conduct the training given the training and validation data. More...

const std::pair< float, float >	TrainOnBatch (int epoch, const Tensor &x, const Tensor &y)
	Train the neural net over one batch of training data.

std::pair< Tensor, Tensor >	Evaluate (const Tensor &x, const Tensor &y, size_t batchsize=128)
	Evaluate the neural net with given data. More...

std::pair< Tensor, Tensor >	EvaluateOnBatch (const Tensor &x, const Tensor &y)
	Evaluate the neural net for one batch of data.

const Tensor	Predict (const Tensor &x, size_t batchsize=128)
	Predict the probability distributation over candicate classes for each data sample. More...

const Tensor	PredictOnBatch (const Tensor &x)
	Predict for one batch data.

const Tensor	Forward (int flag, const Tensor &x)
	Forward layers one by one using the data batch 'x'. More...

const vector< Tensor >	Backward (int flag, const Tensor &grad)
	Backward layers one by one using the gradient batch 'grad'. More...

FeedForwardNet	Clone (std::shared_ptr< Device > device)
	Clone the neuaral net by cloning every layer to the given device. More...

void	ToDevice (std::shared_ptr< Device > device)
	Move the layer data to the given device.

void	ToHost ()

void	AsType (DataType dtype)
	Set the data type of each layer.

std::thread	TrainThread (size_t batchsize, int nb_epoch, const Tensor &x, const Tensor &y, const Tensor &val_x, const Tensor &val_y)
	A wrapper method to spawn a thread to execute Train() method.

std::thread	TrainThread (size_t batchsize, int nb_epoch, const Tensor &x, const Tensor &y)
	A wrapper method to spawn a thread to execute Train() method.

const vector< std::shared_ptr< Layer > >	layers () const

const vector< string >	GetParamNames () const

const vector< ParamSpec >	GetParamSpecs () const

const vector< Tensor >	GetParamValues () const

Protected Attributes
vector< std::shared_ptr< Layer > >	layers_

std::shared_ptr< Updater >	updater_

Loss *	loss_

Metric *	metric_

bool	shuffle_ = true

Device *	device_ = nullptr

DataType	dtype_ = kFloat32

Detailed Description

The feed-forward neural net.

It provides functions for constructing the layers, access layer parameters, and conducting training, evaluation and prediction.

Member Function Documentation

◆ Add() [1/2]

std::shared_ptr<Layer> singa::FeedForwardNet::Add ( std::shared_ptr< Layer > layer )

Add a layer with the assumption that.

this function is called in correct order, i.e., the layers are added following the topological order.
this layer has already been setup (Setup function is called outside). The layer will be freed in the destructor of FeedForwardNet.

◆ Add() [2/2]

std::shared_ptr<Layer> singa::FeedForwardNet::Add	(	const LayerConf &	conf,
		const Shape *	sample_shape = `nullptr`
	)

Add a layer by providing its configuration, and setup it.

Assume the layer is added in corret order. For the first layer, 'sample_shape' (the input sample shape) is necessary for calling Setup().

◆ Backward()

const vector<Tensor> singa::FeedForwardNet::Backward	(	int	flag,
		const Tensor &	grad
	)

Backward layers one by one using the gradient batch 'grad'.

Returns the parameter gradients.

◆ Clone()

FeedForwardNet singa::FeedForwardNet::Clone ( std::shared_ptr< Device > device )

Clone the neuaral net by cloning every layer to the given device.

If 'device' is nullptr, then clone it one the current device.

◆ Compile() [1/2]

void singa::FeedForwardNet::Compile	(	bool	shuffle,
		Optimizer *	opt,
		Loss *	loss,
		Metric *	metric
	)

Set some fields used for training and evaluating the neural net.

This method will instantiate an Updater ,then wrap the Optimier into Updater and always register the parameters of the net instance. If the neural net is constructed for evaluation only, then 'opt' is not necessary; But for training, both 'opt' and 'loss' are necessary. 'shuffle' indicates shuffling training samples within one epoch it is valid using Train(). If to_register is set true, parameter will be registered in Updater.;

◆ Compile() [2/2]

void singa::FeedForwardNet::Compile	(	bool	shuffle,
		bool	to_register,
		std::shared_ptr< Updater >	updater,
		Loss *	loss,
		Metric *	metric
	)

Set some fields used for training and evaluating the neural net.

This method is mainly used in parallel training, where we need multiple neuralnet instances. If the neural net is constructed for evaluation only, then 'updater' is not necessary; But for training, both 'opt' and 'loss' are necessary. 'shuffle' indicates shuffling training samples within one epoch it is valid using Train(). If to_register is set true, parameter will be registered in Updater.;

◆ Evaluate()

std::pair<Tensor, Tensor> singa::FeedForwardNet::Evaluate	(	const Tensor &	x,
		const Tensor &	y,
		size_t	batchsize = `128`
	)

Evaluate the neural net with given data.

Returns one tensor for loss values and one tensor for metric values; Each sample would have a loss value and a metric value (if 'metic' is set in Compile()).'batchsize' is used for controlling the memory footprint. It should be smaller than the total number of samples. Due to memory limit, 'x' and 'y' could not be very large. Hence, it is typically used for small training datasets, e.g., cifar10 and MNIST which can be stored in main memory.

◆ Forward()

const Tensor singa::FeedForwardNet::Forward	(	int	flag,
		const Tensor &	x
	)

Forward layers one by one using the data batch 'x'.

Returns the prediction results (from the last layer).

◆ Predict()

const Tensor singa::FeedForwardNet::Predict	(	const Tensor &	x,
		size_t	batchsize = `128`
	)

Predict the probability distributation over candicate classes for each data sample.

'batchsize' is used for controlling the memory footprint. It should be smaller than the total number of samples. Due to memory limit, 'x' and 'y' could not be very large. Hence, it is typically used for small training datasets, e.g., cifar10 and MNIST which can be stored in main memory.

◆ Train() [1/2]

void singa::FeedForwardNet::Train	(	size_t	batchsize,
		int	nb_epoch,
		const Tensor &	x,
		const Tensor &	y,
		float	val_split = `0.0f`
	)

Conduct the training giving the training data 'x' and label 'y'.

'val_split' of training data is used for validation. Validation is performance before every epoch. Due to memory limit, 'x' and 'y' could not be very large. Hence, it is typically used for small training datasets, e.g., cifar10 and MNIST which can be stored in main memory.

◆ Train() [2/2]

void singa::FeedForwardNet::Train	(	size_t	batchsize,
		int	nb_epoch,
		const Tensor &	x,
		const Tensor &	y,
		const Tensor &	val_x,
		const Tensor &	val_y
	)

Conduct the training given the training and validation data.

Validation is performance before every epoch. Due to memory limit, 'x' and 'y' could not be very large. Hence, it is typically used for small training datasets, e.g., cifar10 and MNIST which can be stored in main memory.

The documentation for this class was generated from the following file:

/home/moaz/incubator-singa/include/singa/model/feed_forward_net.h

Public Member Functions

Protected Attributes