Apache SINGA

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
Apache SINGA
Apache SINGA logo.png
Developer(s)Apache Software Foundation
Initial releaseOctober 8, 2015; 3 years ago (2015-10-08)
Stable release
2.0.0 / April 20, 2019; 2 months ago (2019-04-20)
Written inC++, Python, Java
Operating systemLinux, macOS, Windows
LicenseApache License 2.0
Websitesinga.apache.org

Apache SINGA is an Apache Incubating project for developing an open source machine learning library. It provides a flexible architecture for scalable distributed training, is extensible to run over a wide range of hardware, and has a focus on health-care applications.

History[edit]

The SINGA project was initiated by the DB System Group at National University of Singapore in 2014, in collaboration with the database group of Zhejiang University, in order to support complex analytics at scale, and make database systems more intelligent and autonomic.[1] It focused on distributed deep learning by partitioning the model and data onto nodes in a cluster and parallelize the training.[2][3] The prototype was accepted by Apache Incubator in March 2015. Seven versions have been released as shown in the following table. Since V1.0, SINGA is general to support traditional machine learning models such as logistic regression. Companies like NetEase,[4] yzBigData,Shentilium and others are using SINGA for their applications, including healthcare[5] and finance.

Version Original release date Latest version Release date
Current stable version: 2.0.0 2019-04-20 2.0.0 2019-04-20
Older version, yet still supported: 1.2.0 2018-06-06 1.2.0 2018-06-06
Older version, yet still supported: 1.1.0 2017-02-12 1.1.0 2017-02-12
Older version, yet still supported: 1.0.0 2016-09-08 1.0.0 2016-09-08
Old version, no longer supported: 0.3.0 2016-04-20 0.1.0 2016-04-20
Old version, no longer supported: 0.2.0 2016-01-14 0.2.0 2016-01-14
Old version, no longer supported: 0.1.0 2015-10-08 0.1.0 2015-10-08
Legend:
Old version
Older version, still supported
Latest version
Latest preview version
Future release

Software Stack[edit]

SINGA's software stack includes three major components, namely, core, IO and model. The following figure illustrates these components together with the hardware. The core component provides memory management and tensor operations; IO has classes for reading (and writing) data from (to) disk and network; The model component provides data structures and algorithms for machine learning models, e.g., layers for neural network models, optimizers/initializer/metric/loss for general machine learning models.

Apache Singa software stack

Benchmark for Distributed training[edit]

Workload: we use a deep convolutional neural network, ResNet-50 as the application. ResNet-50 is has 50 convolution layers for image classification. It requires 3.8 GFLOPs to pass a single image (of size 224x224) through the network. The input image size is 224x224.

Hardware: we use p2.8xlarge instances from AWS, each of which has 8 Nvidia Tesla K80 GPUs, 96 GB GPU memory in total, 32 vCPU, 488 GB main memory, 10 Gbps network bandwidth.

Metric: we measure the time per iteration for different number of workers to evaluate the scalability of SINGA. The batch size is fixed to be 32 per GPU. Synchronous training scheme is applied. As a result, the effective batch size is $32N$, where N is the number of GPUs. We compare with a popular open source system which uses the parameter server topology. The first GPU is selected as the server. In the following figure, bars are for the throughput and lines are for the communication cost.

Scalability test

Rafiki[edit]

Rafiki[6] is a sub module of SINGA for providing machine learning analytics service.

Using SINGA[edit]

To get started with SINGA, there are some tutorials available as Jupyter notebooks. The tutorials cover the following:

There is also an online course about SINGA.

See also[edit]

References[edit]

  1. ^ Wei, Wang; Meihui, Zhang; Gang, Chen; H.V., Jagadish; Beng Chin, Ooi; Kian-Lee, Tan; Sheng, Wang (June 2016). "Database Meets Deep Learning: Challenges and Opportunities". SIGMOD Record. 45 (2): 17–22. doi:10.1145/3003665.3003669.
  2. ^ Ooi, Beng Chin; Tan, Kian-Lee; Sheng, Wang; Wang, Wei; Cai, Qingchao; Chen, Gang; Gao, Jinyang; Luo, Zhaojing; Tung, Anthony K. H.; Wang, Yuan; Xie, Zhongle; Zhang, Meihui; Zheng, Kaiping (2015). "SINGA: A distributed deep learning platform" (PDF). ACM Multimedia. doi:10.1145/2733373.2807410. Retrieved 8 September 2016.
  3. ^ Wei, Wang; Chen, Gang; Anh Dinh, Tien Tuan; Gao, Jinyang; Ooi, Beng Chin; Tan, Kian-Lee; Sheng, Wang (2015). "SINGA: putting deep learning in the hands of multimedia users" (PDF). ACM Multimedia. doi:10.1145/2733373.2806232. Retrieved 8 September 2016.
  4. ^ 网易. "网易携手Apache SINGA角逐人工智能新战场_网易科技". tech.163.com. Retrieved 2017-06-03.
  5. ^ "New app allows pre-diabetics to use photos of their meal to check if it is healthy". www.straitstimes.com. Retrieved 6 April 2019.
  6. ^ Wang, Wei; Gao, Jinyang; Zhang, Meihui; Sheng, Wang; Chen, Gang; Khim Ng, Teck; Ooi, Beng Chin; Shao, Jie; Reyad, Moaz (2018). "Rafiki: Machine Learning as an Analytics Service System" (PDF). PVLDB 12(2). doi:10.14778/3282495.3282499. Retrieved 9 January 2019.

External links[edit]