Apache SINGA
A distributed deep learning platform .
 All Classes Namespaces Files Functions Variables Typedefs Enumerator Macros
Public Member Functions | Protected Attributes | List of all members
singa::DataLayer Class Referenceabstract

Base layer for reading records from local Shard, HDFS, lmdb, etc. More...

#include <base_layer.h>

Inheritance diagram for singa::DataLayer:
singa::Layer singa::LMDBDataLayer singa::ShardDataLayer

Public Member Functions

virtual void ComputeFeature (bool training, const vector< SLayer > &srclayers)=0
 Compute features of this layer based on connected layers. More...
 
virtual void Setup (const LayerProto &proto, const vector< SLayer > &srclayers)=0
 Setup layer properties. More...
 
virtual bool is_datalayer () const
 
virtual void ComputeGradient (const vector< SLayer > &srclayers)
 Compute gradients for parameters and connecting layers. More...
 
virtual const vector< Record > & records () const
 
virtual void Setup ()
 Setup layer properties. More...
 
virtual void SetupAfterPartition (const LayerProto &proto, const vector< int > &shape, const vector< SLayer > &srclayers)
 Setup the layer properties except shape. More...
 
virtual void SetupAfterPartition ()
 Setup the layer properties except shape.
 
virtual PartitionType partition_type () const
 
virtual int batchsize () const
 
virtual const Recordsample () const
 
virtual Blob< float > * mutable_data (const Layer *layer)
 
virtual Blob< float > * mutable_grad (const Layer *layer)
 
- Public Member Functions inherited from singa::Layer
virtual void Init (const LayerProto &proto)
 Layer initialization. More...
 
void Init (const Layer &other, const vector< int > &shape)
 Copy layer configuration from the other Layer, and use the shape argument to as its data shape.
 
virtual void ToProto (LayerProto *layer_proto, bool copyData)
 TODO(wangsheng) Marshal layer properties and data into google protobuf object (i.e., snapshot). More...
 
virtual vector< shared_ptr
< Param > > 
GetParams ()
 Layers that have paramters must overload this function. More...
 
virtual void ComputeFeature (bool training)
 
virtual void ComputeGradient ()
 Compute gradients for parameters and connecting layers.
 
virtual int partition_dimension () const
 Decide on which dimension to do the partitioning. More...
 
virtual ConnectionType connection_type (int k) const
 Return connection type between two layers. More...
 
virtual void set_partitionid (int id)
 partition id is the ID of the layer in the original layer.
 
virtual int partitionid () const
 
virtual void set_name (string name)
 
virtual const string type () const
 
const std::string & name () const
 Return name of this layer.
 
const std::string & datablob () const
 
const vector< int > & shape (const Layer *layer) const
 
virtual const Blob< float > & data (const Layer *from) const
 
virtual const Blob< float > & grad (const Layer *from) const
 
virtual const vector< SLayer > srclayers () const
 return LayerS that connected to this layer
 
virtual const vector< SLayer > dstlayers () const
 return LayerS that this layer connected to
 
virtual const int srclayers_size () const
 
virtual const int dstlayers_size () const
 
virtual void ClearDstLayers ()
 
virtual void ClearSrcLayers ()
 
virtual void AddSrcLayer (SLayer src)
 
virtual void AddDstLayer (SLayer dst)
 
virtual bool is_parserlayer () const
 
virtual bool is_losslayer () const
 
virtual bool is_bridgesrclayer () const
 
virtual bool is_bridgedstlayer () const
 

Protected Attributes

bool has_setup_
 
int random_skip_
 
int batchsize_
 
Record sample_
 
vector< Recordrecords_
 
- Protected Attributes inherited from singa::Layer
string name_
 
Blob< float > data_
 
Blob< float > grad_
 
LayerProto layer_proto_
 
vector< SLayer > srclayers_
 
vector< SLayer > dstlayers_
 

Detailed Description

Base layer for reading records from local Shard, HDFS, lmdb, etc.

Cannot be partitioned, always returns kNone for partition type.

Member Function Documentation

virtual void singa::DataLayer::ComputeFeature ( bool  training,
const vector< SLayer > &  srclayers 
)
pure virtual

Compute features of this layer based on connected layers.

Implement forward propagation for BP. TODO(zhaojing) Implement both postive phase and negative phase for CD.

Parameters
trainingtrue if in training phase
srclayerslayers connecting to this layer

Implements singa::Layer.

virtual void singa::DataLayer::ComputeGradient ( const vector< SLayer > &  srclayers)
inlinevirtual

Compute gradients for parameters and connecting layers.

Implement backward propagation for BP. TODO(zhaojing) Calculate gradients for parameters for CD.

Parameters
srclayerslayers connecting to this layer.

Implements singa::Layer.

virtual Blob<float>* singa::DataLayer::mutable_grad ( const Layer from)
inlinevirtual
Returns
a pointer to storing neuron grads of this layer for BP

Reimplemented from singa::Layer.

virtual PartitionType singa::DataLayer::partition_type ( ) const
inlinevirtual
Returns
partition type of this layer, e.g., kNone, kLayer or kData.

Reimplemented from singa::Layer.

virtual void singa::DataLayer::Setup ( const LayerProto proto,
const vector< SLayer > &  srclayers 
)
pure virtual

Setup layer properties.

Setup the shapes for data and parameters, also setup some properties based on the layer configuration and connected src layers.

Parameters
srclayerslayers connecting to this layer

Implements singa::Layer.

Implemented in singa::LMDBDataLayer, and singa::ShardDataLayer.

virtual void singa::DataLayer::Setup ( )
inlinevirtual

Setup layer properties.

Setup the shapes for data and parameters, also setup some properties based on the layer configuration and connected src layers.

Parameters
srclayerslayers connecting to this layer

Reimplemented from singa::Layer.

virtual void singa::DataLayer::SetupAfterPartition ( const LayerProto proto,
const vector< int > &  shape,
const vector< SLayer > &  srclayers 
)
inlinevirtual

Setup the layer properties except shape.

The shape is already set and passed in to set other properties. properties are set according to shapes of itself and connected layers, and configuration. this should not change the current shape_( shape check is done outside the function).

Implements singa::Layer.


The documentation for this class was generated from the following file: