singa-incubating-0.2.0 Release Notes

SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is designed with an intuitive programming model based on the layer abstraction. SINGA supports a wide variety of popular deep learning models.

This release includes the following major features:

• Training on GPU enables training of complex models on a single node with multiple GPU cards.
• Hybrid neural net partitioning supports data and model parallelism at the same time.
• Python wrapper makes it easy to configure the job, including neural net and SGD algorithm.
• RNN model and BPTT algorithm are implemented to support applications based on RNN models, e.g., GRU.
• Cloud software integration includes Mesos, Docker and HDFS.

More details are listed as follows,

• Programming model
  • [SINGA-80] New Blob Level and Address Level Math Operation Interface
  • [SINGA-82] Refactor input layers using data store abstraction
  • [SINGA-87] Replace exclude field to include field for layer configuration
  • [SINGA-110] Add Layer member datavec_ and gradvec_
  • [SINGA-120] Implemented GRU and BPTT (BPTTWorker)

• Neuralnet layers
  • [SINGA-91] Add SoftmaxLayer and ArgSortLayer
  • [SINGA-106] Add dummy layer for test purpose
  • [SINGA-120] Implemented GRU and BPTT (GRULayer and OneHotLayer)

• GPU training support
  • [SINGA-100] Implement layers using CUDNN for GPU training
  • [SINGA-104] Add Context Class
  • [SINGA-105] Update GUN make files for compiling cuda related code
  • [SINGA-98] Add Support for AlexNet ImageNet Classification Model

• Model/Hybrid partition
  • [SINGA-109] Refine bridge layers
  • [SINGA-111] Add slice, concate and split layers
  • [SINGA-113] Model/Hybrid Partition Support

• Python binding
  • [SINGA-108] Add Python wrapper to singa

• Predict-only mode
  • [SINGA-85] Add functions for extracting features and test new data

• Integrate with third-party tools
  • [SINGA-11] Start SINGA on Apache Mesos
  • [SINGA-78] Use Doxygen to generate documentation
  • [SINGA-89] Add Docker support

• Unit test
  • [SINGA-95] Add make test after building

• Other improvment
  • [SINGA-84] Header Files Rearrange
  • [SINGA-93] Remove the asterisk in the log tcp://169.254.12.152:*:49152
  • [SINGA-94] Move call to google::InitGoogleLogging() from Driver::Init() to main()
  • [SINGA-96] Add Momentum to Cifar10 Example
  • [SINGA-101] Add ll (ls -l) command in .bashrc file when using docker
  • [SINGA-114] Remove short logs in tmp directory
  • [SINGA-115] Print layer debug information in the neural net graph file
  • [SINGA-118] Make protobuf LayerType field id easy to assign
  • [SIGNA-97] Add HDFS Store

• Bugs fixed
  • [SINGA-85] Fix compilation errors in examples
  • [SINGA-90] Miscellaneous trivial bug fixes
  • [SINGA-107] Error from loading pre-trained params for training stacked RBMs
  • [SINGA-116] Fix a bug in InnerProductLayer caused by weight matrix sharing