Jump to content

AI accelerator: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
hw accel
→Examples: too many
Line 39: Line 39:


== Examples ==
== Examples ==
{{example farm}}
===Stand alone products===
===Stand alone products===
*Google [[Tensor processing unit]] is an accelerator specifically designed by [[Google]] for its [[TensorFlow]] framework, which is extensively used for [[convolutional neural network]]s. It focuses on a high volume of [[8-bit]] precision arithmetic. The initial first generation focused on inference, while the second generation increased capability for neural network training also.
*Google [[Tensor processing unit]] is an accelerator specifically designed by [[Google]] for its [[TensorFlow]] framework, which is extensively used for [[convolutional neural network]]s. It focuses on a high volume of [[8-bit]] precision arithmetic. The initial first generation focused on inference, while the second generation increased capability for neural network training also.

Revision as of 04:43, 20 November 2017

An AI accelerator is a class of microprocessor[1] or computer system[2] designed to accelerate artificial neural networks, machine vision and other machine learning algorithms for robotics, internet of things and other data-intensive or sensor-driven tasks.[3] They are often manycore designs and generally focus on low-precision arithmetic. A number of vendor-specific terms exist for devices in this space.

History of AI acceleration

Computer systems have frequently complemented the CPU with special purpose accelerators for specialized tasks, most notably video cards for graphics, but also sound cards for sound, etc. As Deep learning and AI workloads rose in prominence, specialize hardware was created or adapted from previous products to accelerate these tasks.

Early attempts

As early as 1993, DSPs have been used as neural network accelerators e.g. to accelerate OCR software,[4]. In the 1990s there were also attempts to create parallel high throughput systems for workstations aimed at various applications, including neural network simulations.[5][6][7] FPGA-based accelerators were also first explored in the 1990s for both inference[8] and training[9]. ANNA was a neural net CMOS accelerator developed by Yann LeCun.[10]

Heterogeneous computing

Heterogeneous computing began the incorporation of a number of specialized processors in a single system, or even a single chip, each optimized for a specific type of task. Architectures such as the Cell microprocessor[11] have features significantly overlapping with AI accelerators including: support for packed low precision arithmetic, dataflow architecture, and prioritising 'throughput' over latency. The Cell microprocessor would go on to be applied to a number of tasks[12][13][14] including AI.[15][16][17]

CPUs themselves also gained increasingly wide SIMD units (driven by video and gaming workloads) and support for packed low precision data types.[18]

Use of GPGPU

Graphics processing units or GPUs are specialized hardware for the manipulation of images. As the mathematical basis of neural networks and image manipulation are similar, embarassingly parallel tasks involving matrices, GPUs became increasingly used for machine learning tasks.[19][20][21]As such, as of 2016 GPUs are popular for AI work, and they continue to evolve in a direction to facilitate deep learning, both for training[22] and inference in devices such as self-driving cars.[23] - and gaining additional connective capability for the kind of dataflow workloads AI benefits from (e.g. Nvidia NVLink).[24] As GPUs have been increasingly applied to AI acceleration, GPU manufacturers have incorporated neural network specific hardware to further accelerate these tasks.[25] Tensor cores are intended to speed up the training of neural networks.[25]

Use of FPGA

Deep learning frameworks are still evolving, making it hard to design custom hardware. Reconfigurable devices like field-programmable gate arrays (FPGA) make it easier to evolve hardware, frameworks and software alongside each other.[8][9][26]

Microsoft has used FPGA chips to accelerate inference.[27][28] The application of FPGAs to AI acceleration has also motivated Intel to purchase Altera with the aim of integrating FPGAs in server CPUs, which would be capable of accelerating AI as well as general purpose tasks.[citation needed]

Emergence of dedicated AI accelerator ASICs

Whilst GPUs and FPGAs perform far better than CPUs for these AI related tasks, a factor of 10 in efficiency[29][30] can still be gained with a more specific design, via an application-specific integrated circuit (ASIC).[citation needed] These include differences in memory use [citation needed] and the use of lower precision numbers.[31][32]

Nomenclature

As of 2016, the field is still in flux and vendors are pushing their own marketing term for what amounts to an "AI accelerator", in the hope that their designs and APIs will dominate. There is no consensus on the boundary between these devices, nor the exact form they will take, however several examples clearly aim to fill this new space, with a fair amount of overlap in capabilities.

In the past when consumer graphics accelerators emerged, the industry eventually adopted Nvidia's self-assigned term, "the GPU",[33] as the collective noun for "graphics accelerators", which had taken many forms before settling on an overall pipeline implementing a model presented by Direct3D.

Examples

Stand alone products

GPU based products

  • Nvidia Tesla is Nvidia's line of GPU derived products marketed for GPGPU and AI tasks.
    • Nvidia Volta is a microarchitecture which augments the Graphics processing unit with additional 'tensor units' targeted specifically at accelerating calculations for neural networks[37]
    • Nvidia DGX-1 is a Nvidia workstation/server product which incorporates Nvidia brand GPUs for GPGPU tasks including machine learning.[38]
  • Radeon Instinct is AMD's line of GPU derived products for AI acceleration.[39]

AI accelerating co-processors

Research and unreleased products

  • Eyeriss is a accelerator design aimed explicitly at convolutional neural networks, using a scratchpad and on chip network architecture.[44]
  • Kalray is a accelerator for convolutional neural nets.[45]
  • SpiNNaker is a many-core design specialized for simulating a large neural network.
  • Graphcore IPU is a graph-based AI accelerator.[46]
  • DPU, by wave computing, a dataflow architecture[47]
  • STMicroelectronics at the start of 2017 presented a demonstrator SoC manufactured in a 28 nm process containing a deep CNN accelerator.[48]
  • NM500 is the latest as of 2016 in a series of accelerator chips for Radial Basis Function neural nets from General Vision.[49]
  • TrueNorth is a manycore design based on spiking neurons rather than traditional arithmetic.[50][51]
  • Intel Loihi is an experimental neuromorphic chip.[52]
  • BrainChip in September 2017 introduced a commercial PCI Express card with a Xilinx Kintex Ultrascale FPGA running neuromorphic neural cores applying pattern recognition on 600 video images per second using 16 watts of power.[53]
  • IIT Madras is designing a spiking neuron accelerator for big-data analytics.[54]

Potential applications

See also

References

  1. ^ "Intel unveils Movidius Compute Stick USB AI Accelerator".
  2. ^ "Inspurs unveils GX4 AI Accelerator".
  3. ^ "google developing AI processors".google using its own AI accelerators.
  4. ^ "convolutional neural network demo from 1993 featuring DSP32 accelerator".
  5. ^ "design of a connectionist network supercomputer".
  6. ^ "The end of general purpose computers (not)".This presentation covers a past attempt at neural net accelerators, notes the similarity to the modern SLI GPGPU processor setup, and argues that general purpose vector accelerators are the way forward (in relation to RISC-V hwacha project. Argues that NN's are just dense and sparse matrices, one of several recurring algorithms)
  7. ^ "SYNAPSE-1: a high-speed general purpose parallel neurocomputer system".
  8. ^ a b "Space Efficient Neural Net Implementation" (PDF).
  9. ^ a b "A Generic Building Block for Hopfield Neural Networks with On-Chip Learning" (PDF).
  10. ^ Application of the ANNA Neural Network Chip to High-Speed Character Recognition
  11. ^ "Synergistic Processing in Cell's Multicore Architecture".
  12. ^ "Performance of Cell processor for biomolecular simulations" (PDF).
  13. ^ "Video Processing and Retreival on Cell architecture".
  14. ^ "Ray Tracing on the Cell Processor".
  15. ^ "Development of an artificial neural network on a heterogeneous multicore architecture to predict a successful weight loss in obese individuals" (PDF).
  16. ^ "Parallelization of the Scale-Invariant Keypoint Detection Algorithm for Cell Broadband Engine Architecture".
  17. ^ "Data Mining Algorithms on the Cell Broadband Engine".
  18. ^ "Improving the performance of video with AVX".
  19. ^ "microsoft research/pixel shaders/MNIST".
  20. ^ "how the gpu came to be used for general computation".
  21. ^ "imagenet classification with deep convolutional neural networks" (PDF).
  22. ^ "nvidia driving the development of deep learning".
  23. ^ "nvidia introduces supercomputer for self driving cars".
  24. ^ "how nvlink will enable faster easier multi GPU computing".
  25. ^ a b Harris, Mark (May 11, 2017). "CUDA 9 Features Revealed: Volta, Cooperative Groups and More". Retrieved August 12, 2017.
  26. ^ "FPGA Based Deep Learning Accelerators Take on ASICs". The Next Platform. 2016-08-23. Retrieved 2016-09-07.
  27. ^ "microsoft extends fpga reach from bing to deep learning".
  28. ^ "Accelerating Deep Convolutional Neural Networks Using Specialized Hardware" (PDF).
  29. ^ "Google boosts machine learning with its Tensor Processing Unit". 2016-05-19. Retrieved 2016-09-13.
  30. ^ "Chip could bring deep learning to mobile devices". www.sciencedaily.com. 2016-02-03. Retrieved 2016-09-13.
  31. ^ "Deep Learning with Limited Numerical Precision" (PDF).
  32. ^ Rastegari, Mohammad; Ordonez, Vicente; Redmon, Joseph; Farhadi, Ali (2016). "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks". arXiv:1603.05279 [cs.CV].
  33. ^ "NVIDIA launches he Worlds First Graphics Processing Unit, the GeForce 256,".
  34. ^ Kampman, Jeff (17 October 2017). "Intel unveils purpose-built Neural Network Processor for deep learning". Tech Report. Retrieved 18 October 2017.
  35. ^ "Intel Nervana Neural Network Processors (NNP) Redefine AI Silicon". Retrieved 20 October 2017.
  36. ^ "The Evolution of EyeQ".
  37. ^ "Nvidia goes beyond the GPU for AI with Volta".
  38. ^ "nvidia dgx-1" (PDF).
  39. ^ Smith, Ryan (12 December 2016). "AMD Announces Radeon Instinct: GPU Accelerators for Deep Learning, Coming in 2017". Anandtech. Retrieved 12 December 2016.
  40. ^ "The highest performance neural network inference accelerator".
  41. ^ "The iPhone X's new neural engine exemplifies Apple's approach to AI". The Verge. Retrieved 2017-09-23.
  42. ^ "Cadence Unveils Industry's First Neural Network DSP IP for Automotive, Surveillance, Drone and Mobile Markets".
  43. ^ "HUAWEI Reveals the Future of Mobile AI at IFA 2017".
  44. ^ Chen, Yu-Hsin; Krishna, Tushar; Emer, Joel; Sze, Vivienne (2016). "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks". IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers. pp. 262–263. {{cite web}}: Cite has empty unknown parameter: |1= (help)
  45. ^ "kalray MPPA" (PDF).
  46. ^ "Graphcore Technology".
  47. ^ "Wave Computing's DPU architecture".
  48. ^ "A 2.9 TOPS/W Deep Convolutional Neural Network SoC in FD-SOI 28nm for Intelligent Embedded Systems" (PDF).
  49. ^ "NM500, Neuromorphic chip with 576 neurons".
  50. ^ "yann lecun on IBM truenorth".argues that spiking neurons have never produced leading quality results, and that 8-16 bit precision is optimal, pushes the competing 'neuflow' design
  51. ^ "IBM cracks open new era of neuromorphic computing". TrueNorth is incredibly efficient: The chip consumes just 72 milliwatts at max load, which equates to around 400 billion synaptic operations per second per watt — or about 176,000 times more efficient than a modern CPU running the same brain-like workload, or 769 times more efficient than other state-of-the-art neuromorphic approaches
  52. ^ "Intel's New Self-Learning Chip Promises to Accelerate Artificial Intelligence".
  53. ^ "BrainChip Accelerator".
  54. ^ "India preps RISC-V Processors - Shakti targets servers, IoT, analytics". The Shakti project now includes plans for at least six microprocessor designs as well as associated fabrics and an accelerator chip
  55. ^ "drive px".
  56. ^ "design of a machine vision system for weed control" (PDF).
  57. ^ "qualcomm research brings server class machine learning to every data devices".
  58. ^ "movidius powers worlds most intelligent drone".
Retrieved from "https://en.wikipedia.org/w/index.php?title=AI_accelerator&oldid=811207849"