NVLink

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
NVLink
NVidia NVLink two lines of text.png
ManufacturerNvidia
TypeMulti-GPU and CPU
PredecessorScalable Link Interface

NVLink is a wire-based communications protocol serial multi-lane near-range communication link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices use mesh networking to communicate instead of a central Hub. The protocol was first announced in March 2014 and uses a proprietary High-Speed Signaling interconnect (NVHS).[1]

Principle[edit]

NVLink is a wire-based communications protocol for near-range semiconductor communications developed by Nvidia that can be used for data and control code transfers in processor systems between CPUs and GPUs and solely between GPUs. NVLink specifies a point-to-point connections with data rates of 20 and 25 Gbit/s (v1.0/v2.0) per data lane per direction. Total data rates in real world systems are 160 and 300 GByte/s (v1.0/v2.0) for the total system sum of input and output data streams.[2] NVLink products introduced to date focus on the high-performance application space.

Performance[edit]

The following table shows a basic metrics comparison based upon standard specifications:

Interconnect Transfer
Rate
Line Code Eff. Payload Rate
per Lane
per Direction
Max total
Lane Length
(PCIe: incl. 5" for PCBs)
PCIe 1.x 2 GT/s 8b/10b ~0.2 GB/s 20" = ~51 cm
PCIe 2.x 4 GT/s 8b/10b ~0.4 GB/s 20" = ~51 cm
PCIe 3.x 8 GT/s 128b/130b ~1 GB/s 20" = ~51 cm[3]
PCIe 4.0 16 GT/s 128b/130b ~2 GB/s 8-12" = ~20-30 cm[4]
PCIe 5.0 32 GT/s[5] 128b/130b ~4 GB/s
NVLink 1.0 20 GT/s ~2.5 GB/s
NVLink 2.0 25 GT/s ~3.125 GB/s

The following table shows a comparison of relevant bus parameters for real world semiconductors that all offer NVLink as one of their options:

Semiconductor Board/Bus
delivery variant
Interconnect Transmission
Technology
Rate (per lane)
Lanes per
Sub-Link
(out + in)
Sub-Link Data Rate
(per data direction)
Sub-Link
or Unit
Count
Total Data Rate
(out + in)
Total
Lanes
(out + in)
Total
Data Rate
(out + in)
Nvidia GP100 P100 SXM[6],
P100 PCI-E[7]
PCIe 3.0 08 GT/s 16 + 16 128 Gbit/s = 16 GByte/s 1 016 + 016 GByte/s[8] 32 032 GByte/s
Nvidia GV100 V100 SXM2[9],
V100 PCI-E[10]
PCIe 3.0 08 GT/s 16 + 16 128 Gbit/s = 16 GByte/s 1 016 + 016 GByte/s 32 032 GByte/s
Nvidia TU104 GeForce RTX 2080,
Quadro RTX 5000
PCIe 3.0 08 GT/s 16 + 16 128 Gbit/s = 16 GByte/s 1 016 + 016 GByte/s 32 032 GByte/s
Nvidia TU102 GeForce RTX 2080 Ti,
Quadro RTX 6000/8000
PCIe 3.0 08 GT/s 16 + 16 128 Gbit/s = 16 GByte/s 1 016 + 016 GByte/s 32 032 GByte/s
Nvidia Xavier[11] (generic) PCIe 4.0 Ⓓ
2 units: x8 (dual)
1 unit: x4 (dual)
3 units: x1[12][13]
16 GT/s
08 + 08
04 + 04
1 + 010

128 Gbit/s = 16 GByte/s
64 Gbit/s = 08 GByte/s
16 Gbit/s = 02 GByte/s

2
1
3

032 + 032 GByte/s
008 + 008 GByte/s
006 + 006 GByte/s
40 80 GByte/s
IBM Power9[14] (generic) PCIe 4.0 16 GT/s 16 + 16 256 Gbit/s = 32 GByte/s 3 096 + 096 GByte/s 96 192 GByte/s
Nvidia GP100 P100 SXM,
(not available with P100 PCI-E)[15]
NVLink 1.0 20 GT/s 08 + 08 160 Gbit/s = 20 GByte/s 4 080 + 080 GByte/s 64 160 GByte/s
Nvidia Xavier (generic) NVLink 1.0[16] 20 GT/s[17] 08 + 08 160 Gbit/s = 20 GByte/s[18]
IBM Power8+ (generic) NVLink 1.0 20 GT/s 08 + 08 160 Gbit/s = 20 GByte/s 4 080 + 080 GByte/s 64 160 GByte/s
Nvidia GV100 V100 SXM2[19]
(not available with V100 PCI-E)
NVLink 2.0 25 GT/s 08 + 08 200 Gbit/s = 25 GByte/s 6[20] 150 + 150 GByte/s 96 300 GByte/s
IBM Power9[21] (generic) NVLink 2.0
(BlueLink ports)
25 GT/s 08 + 08 200 Gbit/s = 25 GByte/s 6 150 + 150 GByte/s 96 300 GByte/s
NVSwitch[22] (generic)
(fully connected 18x18 switch)
NVLink 2.0 25 GT/s 08 + 08 200 Gbit/s = 25 GByte/s 2 * 8 + 2
= 18
450 + 450 GByte/s 288 900 GByte/s
Nvidia TU104 GeForce RTX 2080,
Quadro RTX 5000[23]
NVLink 2.0 25 GT/s 08 + 08 200 Gbit/s = 25 GByte/s 1 025 + 025 GByte/s 16 050 GByte/s
Nvidia TU102 GeForce RTX 2080 Ti,
Quadro RTX 6000/8000[23]
NVLink 2.0 25 GT/s 08 + 08 200 Gbit/s = 25 GByte/s 2 050 + 050 GByte/s 32 100 GByte/s

Note: Data Rate columns were rounded by being approximated by transmission rate, see real world performance paragraph

: sample value; NVLink sub-link bundling should be possible
: sample value; other fractions for the PCIe lane usage should be possible
: a single (no! 16) PCIe lane transfers data over a differential pair
: various limitations of finally possible combinations might apply due to chip pin muxing and board design
dual: interface unit can either be configured as a root hub or an end point
generic: bare semiconductor without any board design specific restrictions applied

Real world performance could be determined by applying different encapsulation taxes as well usage rate. Those come from various sources:

  • 128b/130b line code (see e.g. PCI Express data transmission for versions 3.0 and higher)
  • Link control characters
  • Transaction header
  • Buffering capabilities (depends on device)
  • DMA usage on computer side (depends on other software, usually negligible on benchmarks)

Those physical limitations usually reduce the data rate to between 90 and 95% of the transfer rate. NVLink benchmarks show an achievable transfer rate of about 35.3 GB/s (host to device) for a 40 GB/s (2 sub-lanes uplink) NVLink connection towards a P100 GPU in a system that is driven by a set of IBM Power8 CPUs.[24]

Usage with Plug-In Boards[edit]

For the various versions of plug-in boards (a yet small number of high-end gaming and professional graphics GPU boards with this feature exist) that are exposing extra connectors for joining them into a NVLink group a similar amount of slightly varying, relatively compact, PCB based interconnection plugs does exist. Typically only boards of same type will mate together due to their physical and logical design. For some setups two identical plugs need to be applied for achieving the full data rate. As of now the typical plug is U-shaped with a fine grid edge connector on each of the end strokes of the shape facing away from the viewer. The wide of the plugs determines how far away the plug-in cards need to be seated to the main board of the hosting computer system - a distance of for the placement of the card is commonly determined by the matching plug (known available plug widths are 3 to 5 and also depending on board type).[25][26] The interconnect is often referred as SLI (Scalable Link Interface) from 2004 for its structural design and appearance even if the modern NVLink based design is of a quite different technical nature with different features in its basic levels compared to the former design. Reported real world devices are:[27]

  • Quadro GP100 (a pair of cards will make use of up to 2 bridges[28]; the setup realizes either 2 or 4 NVLink connections with up to 160 GB/s[29] - this might resemble NVLink 1.0 with 20 GT/s)
  • Quadro GV100 (a pair of cards will need up to 2 bridges and realize up to 200 GB/s[30] - this might resemble NVLink 2.0 with 25 GT/s and 4 links)
  • GeForce RTX 2080 based on TU104 (with single bridge "GeForce RTX NVLink-Bridge"[31])
  • GeForce RTX 2080 Ti based on TU102 (with single bridge "GeForce RTX NVLink-Bridge"[32])
  • Quadro RTX 5000[33] based on TU104[34] (with single bridge "NVLink" up to 50 GB/s[35] - this might resemble NVLink 2.0 with 25 GT/s and 1 link)
  • Quadro RTX 6000[36] based on TU102[37] (with single bridge "NVLink HB" up to 100 GB/s[38] - this might resemble NVLink 2.0 with 25 GT/s and 2 links)
  • Quadro RTX 8000[39] based on TU102[40] (with single bridge "NVLink HB" up to 100 GB/s[41] - this might resemble NVLink 2.0 with 25 GT/s and 2 links)

Service Software and Programming[edit]

By means of the NVML-API (NVIDIA Management Library) offers for the Tesla, Quadro and Grid line of products NVidia a set of functions for programmatically controlling some aspects of NVLink interconnects on Windows and Linux systems, such as component evaluation and versions along with status/error querying and performance monitoring.[42] Further with the provision of the NCCL library (NVIDIA collective communications library) developers in the public space shall be enabled for realizing e.g. powerful implementations for artificial intelligence and similar computation hungry topics atop NVLink.[43] The page "3D Settings" => "Configure SLI, Surround, PhysX" from the NVidia Control panel and the CUDA sample application "simpleP2P" are supposed using such APIs as mentioned upfront for realizing their services in respect to their NVLink features. At least on the Linux platform the command line application with a certain sub-command "nvidia-smi nvlink" provides a similar set of advanced information and control.[44]

History[edit]

On 5 April 2016, Nvidia announced that NVLink would be implemented in the Pascal-microarchitecture-based GP100 GPU, as used in, for example, Nvidia Tesla P100 products.[45] With the introduction of the DGX-1 high performance computer base it was possible to have up to eight P100 modules in a single rack system connected to up to two host CPUs. The carrier board (...) allows for a dedicated board for routing the NVLink connections – each P100 requires 800 pins, 400 for PCIe + power, and another 400 for the NVLinks, adding up to nearly 1600 board traces for NVLinks alone (...).[46] Each CPU has direct connection to 4 units of P100 via PCIe and each P100 has one NVLink each to the 3 other P100s in the same CPU group plus one more NVLink to one P100 in the other CPU group. Each NVLink (link interface) offers a bidirectional 20 GB/sec up 20 GB/sec down, with 4 links per GP100 GPU, for an aggregate bandwidth of 80 GB/sec up and another 80 GB/sec down.[47] NVLink supports routing so that in the DGX-1 design for every P100 a total of 4 of the other 7 P100s are directly reachable and the remaining 3 are reachable with only one hop. According to depictions in Nvidia's blog based publications from 2014 NVLink allows bundling of individual links for increased point to point performance so that for example a design with two P100s and all links established between the two units would allow the full NVLink bandwidth of 80 GB/s between them.[48]

At GTC2017, Nvidia presented its Volta generation of GPUs and indicated the integration of a revised version 2.0 of NVLink that would allow total i/o data rates of 300 GB/s for a single chip for this design, and further announced the option for pre-orders with a delivery promise for Q3/2017 of the DGX-1 and DGX-Station high performance computers that will be equipped with GPU modules of type V100 and have NVLink 2.0 realized in either a networked (two groups of four V100 modules with inter-group connectivity) of or a fully interconnected fashion of one group of four V100 modules.

In 2017-2018, IBM and Nvidia delivered two supercomputers for the US Department of Energy named "Summit" and "Sierra",[49] which combine IBM's POWER9 family of CPUs and Nvidia's Volta architecture, using NVLink 2.0 for the CPU-GPU and GPU-GPU interconnects and InfiniBand EDR for the system interconnects.[50]

See also[edit]

References[edit]

  1. ^ Nvidia NVLINK 2.0 arrives in IBM servers next year by Jon Worrel on fudzilla.com on August 24, 2016
  2. ^ "What Is NVLink?". Nvidia. 2014-11-14.
  3. ^ https://www.elektronik-kompendium.de/sites/com/0904051.htm
  4. ^ https://www.elektronik-kompendium.de/sites/com/0904051.htm
  5. ^ https://www.tomshardware.com/news/pcie-4.0-5.0-pci-sig-specification,38460.html
  6. ^ https://www.heise.de/preisvergleich/nvidia-tesla-p100-sxm2-nvtp100-sxm-a1501151.html
  7. ^ https://www.heise.de/preisvergleich/pny-tesla-p100-pcie-tcsp100m-16gb-pb-nvtp100-16-a1501119.html
  8. ^ NVLink Takes GPU Acceleration To The Next Level by Timothy Prickett Morgan at nextplatform.com on May 4, 2016
  9. ^ https://www.techpowerup.com/gpu-specs/tesla-v100-sxm2-16-gb.c3018
  10. ^ https://www.heise.de/preisvergleich/pny-quadro-gv100-vcqgv100-pb-a1800874.html
  11. ^ Tegra Xavier - Nvidia at wikichip.org
  12. ^ JETSON AGX XAVIER PLATFORM ADAPTATION AND BRING-UP GUIDE "Tegra194 PCIe Controller Features" on page 14; stored at arrow.com
  13. ^ How to enable PCIe x2 slot with Xavier? on devtalk.nvidia.com
  14. ^ POWER9 Webinar presentation by IBM for Power Systems VUG by Jeff Stuecheli on January 26, 2017
  15. ^ All aboard the PCIe bus for Nvidia's Tesla P100 supercomputer grunt by Chris Williams at theregister.co.uk on June 20, 2016
  16. ^ Tegra Xavier - Nvidia at wikichip.org
  17. ^ Tegra Xavier - Nvidia at wikichip.org
  18. ^ https://blogs.nvidia.com/blog/2018/11/13/xavier-milestone-safe-self-driving/
  19. ^ https://www.heise.de/newsticker/meldung/Nvidia-Tesla-V100-PCIe-Steckkarte-mit-Volta-Grafikchip-und-16-GByte-Speicher-angekuendigt-3753051.html
  20. ^ GV100 Blockdiagramm in "GTC17: NVIDIA präsentiert die nächste GPU-Architektur Volta - Tesla V100 mit 5.120 Shadereinheiten und 16 GB HBM2" by Andreas Schilling on hardwareluxx.de on May 10, 2017
  21. ^ NVIDIA Volta GV100 GPU Chip For Summit Supercomputer Twice as Fast as Pascal P100 – Speculated To Hit 9.5 TFLOPs FP64 Compute by Hassan Mujtaba at wccftech.com on December 20, 2016
  22. ^ http://images.nvidia.com/content/pdf/nvswitch-technical-overview.pdf
  23. ^ a b Angelini, Chris (14 September 2018). "Nvidia's Turing Architecture Explored: Inside the GeForce RTX 2080". Tom's Hardware. p. 7. Retrieved 28 February 2019. TU102 and TU104 are Nvidia’s first desktop GPUs rocking the NVLink interconnect rather than a Multiple Input/Output (MIO) interface for SLI support. The former makes two x8 links available, while the latter is limited to one. Each link facilitates up to 50 GB/s of bidirectional bandwidth. So, GeForce RTX 2080 Ti is capable of up to 100 GB/s between cards and RTX 2080 can do half of that.
  24. ^ Comparing NVLink vs PCI-E with NVIDIA Tesla P100 GPUs on OpenPOWER Servers by Eliot Eshelman on microway.com on January 26, 2017
  25. ^ https://www.nvidia.com/de-de/design-visualization/nvlink-bridges/
  26. ^ https://www.nvidia.com/de-de/geforce/graphics-cards/rtx-2080-ti/#sli-bridges
  27. ^ https://www.pugetsystems.com/labs/articles/NVLink-on-NVIDIA-GeForce-RTX-2080-2080-Ti-in-Windows-10-1253/
  28. ^ http://www.shopblt.com/cgi-bin/shop/shop.cgi?action=enter&thispage=011004001508_B2PU977P.shtml
  29. ^ https://www.hardwareluxx.de/index.php/news/hardware/grafikkarten/41825-nvidia-praesentiert-quadro-gp100-mit-gp100-gpu-und-16-gb-hbm2.html
  30. ^ https://www.nvidia.com/de-de/design-visualization/nvlink-bridges/
  31. ^ https://www.nvidia.com/de-de/geforce/graphics-cards/rtx-2080/#sli-bridges
  32. ^ https://www.nvidia.com/de-de/geforce/graphics-cards/rtx-2080-ti/#sli-bridges
  33. ^ https://www.nvidia.com/en-us/design-visualization/quadro-desktop-gpus/
  34. ^ https://www.hardwareinside.de/nvidia-quadro-rtx-6000-und-rtx-5000-ready-fuer-pre-order-36399/
  35. ^ https://www.pny.com/promo/nvlink
  36. ^ https://www.nvidia.com/en-us/design-visualization/quadro-desktop-gpus/
  37. ^ https://www.hardwareinside.de/nvidia-quadro-rtx-6000-und-rtx-5000-ready-fuer-pre-order-36399/
  38. ^ https://www.pny.com/promo/nvlink
  39. ^ https://www.nvidia.com/en-us/design-visualization/quadro-desktop-gpus/
  40. ^ https://www.techpowerup.com/gpu-specs/quadro-rtx-8000.c3306
  41. ^ https://www.pny.com/promo/nvlink
  42. ^ https://docs.nvidia.com/deploy/nvml-api/group__NvLink.html#group__NvLink
  43. ^ https://developer.nvidia.com/nccl
  44. ^ https://www.pugetsystems.com/labs/articles/NVLink-on-NVIDIA-GeForce-RTX-2080-2080-Ti-in-Windows-10-1253/
  45. ^ "Inside Pascal: NVIDIA's Newest Computing Platform". 2016-04-05.
  46. ^ Anandtech.com
  47. ^ NVIDIA Unveils the DGX-1 HPC Server: 8 Teslas, 3U, Q2 2016 by anandtech.com on April, 2016
  48. ^ How NVLink Will Enable Faster, Easier Multi-GPU Computing by Mark Harris on November 14, 2014
  49. ^ "Whitepaper: Summit and Sierra Supercomputers" (PDF). 2014-11-01.
  50. ^ "Nvidia Volta, IBM POWER9 Land Contracts For New US Government Supercomputers". AnandTech. 2014-11-17.