Network Device Interface

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

Network Device Interface (NDI) is a royalty-free software standard developed by NewTek to enable video-compatible products to communicate, deliver, and receive broadcast quality video in a high quality, low latency manner that is frame-accurate and suitable for switching in a live production environment.

Technology[edit]

NDI was developed by NewTek, and is available to anyone with a royalty-free license. NewTek has an SDK available for Windows, Linux and macOS.[1] NDI has also been ported to iOS, tvOS, Android, Raspberry Pi, and FPGA.[2][3]

NDI is designed to run over existing Gigabit networks,[4] with the NDI codec[5] expected to deliver 1080i HD video at VBR data rates typically around 100 Mbit/s.[6]

NDI uses the mDNS (Bonjour / Zeroconf) discovery mechanism to advertise sources on a local area network, such that NDI receiving devices can automatically discover and offer those sources. Sources are created using an arbitrarily selected TCP port from a range of ports on the NDI send host. When a source is requested a TCP connection is established on the appropriate port with the NDI receiver connecting to the NDI sender. NDI 3.x has options to use UDP Multicast or Unicast with FEC(Forward error correction) instead of TCP, and can load balance streams across multiple Gigabit NICs without link aggregation.

NDI carries video, multichannel uncompressed audio and metadata. Metadata messages can be sent in both directions allowing the sender and receiver to message one another over the connection with arbitrary metadata in XML form.[7] This directional metadata system allows for functionality such as active tally information fed back to sources to understand that they are on-air (program / preview). NDI also allows senders to determine the number of connected receivers, so they can skip unnecessary processing and network bandwidth utilisation when there are no NDI receiver clients connected. NDI Receivers can opt to connect to various combinations of streams, to support things like audio-only or metadata-only connections where video is not required.

Comparison of common IP Video protocols[edit]

Other IP Video protocols emerging for use in professional video production (rather than IP Video used for distribution to end users) include SMPTE 2022, SMPTE2110, ASPEN (which appears to have been largely superseded by SMPTE2110) and Sony NMI. There are clear differences in the technology used by these protocols.

Parameter NDI NDI HX NDI HX2 SMPTE 2022-6[8] SMPTE_2110[8] ASPEN[9] NMI[10]
Transport TCP/UDP/Multi-TCP** UDP (TCP) UDP, Multi TCP UDP UDP UDP UDP
Image Format Size / Aspect Independent Size / Aspect Independent Size / Aspect Independent SDI Formats only Size / Aspect Independent
Tally Feedback Yes Yes Yes No No*** No
Bidirectional Device Control Yes Yes Yes No No No
Integrated Alpha Channel Yes No No No Yes
Compression NDI Codec (SHQ 2/7)[11] NDI HX (h.264) h.264 NONE * (proposed) NONE NONE / LLVC Codec
Connection Socket, Unicast / Multicast and FEC Unicast/MultiCast Unicast/MultiCast MultiCast MultiCast MultiCast MultiCast / ?
HD (1080i) Data Rate ~ 100 Mbit/s 8-20 Mbit/s ~1-50 Mbit/s >1.5 Gbit/s >1.1 Gbit/s >1.5 Gbit/s >1.5 Gbit/s / up to 14:1[12]
Essence Packing Discrete Audio, Metadata and Video Frame packets, single connection Delivered as Discrete Audio, Metadata and Video Frame packets, single connection Discrete Audio, Metadata and Video Frame packets, single connection Packetized Raw SDI bitstream Discrete A, V, Essence and Metadata on separate connections with different protocols Multiple MPEG Transport Streams Frame Aligned 2022-6 / LLVC
Infrastructure Gigabit / Wireless / Load Balanced Multi NIC / 10GBit Gigabit / Wireless Gigabit / Wireless req. 10GBit req. 10GBit req. 10GBit req. 10GBit / Gigabit
Service Discovery Bonjour (mDNS), NDI Access (manual), Server (NDI4) Bonjour (mDNS) & NDI Access (manual) Bonjour (mDNS), NDI Access (manual), Server (NDI4) NMOS[13] proposed AMWA IS-04 JSON-RPC Plug & Play (NDCP)
API free license, SDK Libraries for Win(x86), Mac, Linux(x86 & ARM), iOS[14], FPGA reference hardware encode, Decode with NDI Libraries Send with NDI Embedded SDK, Receive with Free NDI Libraries paid-for SMPTE Standards paid-for SMPTE Standards paid-for SMPTE RDD

*The TICO codec can be used to compress UHD by 4:1 so an encoded stream can be carried along a SMPTE 2022-6 channel at the same uncompressed bandwidth as HD.[15] SMPTE 2110 with TR-03 also offers the potential to use TICO.[16] This requires a proprietary encoder and decoder which are generally implemented as silicon on each end. * * NDI v1.0 was pure TCP. Later versions included options for UDP Unicast and Multicast as appropriate and FEC. NDI 4.0 adds 'Multi-TCP' transport. NDI HX is generally UDP but uses TCP control commands. * * * Tally for SMPTE 2110 relies on external data sources, using proposed AMWA IS 07.

History[edit]

NDI was publicly revealed by NewTek on 8 September 2015 and was demonstrated at the IBC broadcast exhibition in Amsterdam that week.[17] The first device shown using NDI was the NewTek TriCaster which delivered an NDI feed from each of its SDI inputs as well as 4 output feeds from its vision mixer. TriCaster could also receive up to 2 NDI sources from other devices (increased to 12 in later releases and up to 44 in NewTek's IP Series ).[18]

NDI devices from other vendors followed during 2016. The first 3rd party products came from Gallery Sienna and included an NDI signal generator for macOS, a desktop scan converter for macOS and the NDICam camera app for iPhone which delivers a native NDI stream from iOS devices.[19][20]

NewTek had previously created a predecessor of NDI called AirSend to get video from external devices into their TriCaster products. AirSend had been implemented by a number of CG manufacturers including VizRT and Chyron. In order to quickly bring these products into the NDI space, NewTek created a new driver to replace the existing AirSend driver, which could be installed on these existing AirSend compatible devices, instantly converting them to NDI compatible devices with no change required by the original CG vendors.[21]

Another early adopter of NDI was VMix, a Windows-based vision mixer which offers NDI inputs and outputs.[22] A significant increase in the NDI installed base came when live streaming application XSplit added support for NDI.[23]

Later in 2016, NewTek delivered NDI 2.0 which added features including support for service discovery across subnets. In the first half of 2017, BirdDog began demonstrating and shipping BirdDog Studio NDI - a PoE powered portable FPGA based NDI encoder with HDMI and SDI inputs with Tally, supporting up to 1080p60 video resolutions.[3] On 12 July 2017 NewTek announced NDI 3.0 which added multicast, NDI/HX and other new features, introducing support for specific PTZ Cameras with H.264 chipsets and updated firmware.[24]

In April 2018 at the NAB Show, Microsoft announced they had added NDI support to Skype for Content Creators.[25] Other announcements at NAB2018 included NDI 3.5, and new NDI support from vendors including EVS[26] and Avid.[27]

In April 2019 ahead of NAB NewTek announced the addition of Multi-TCP mode to NDI 4.0 which is reported to leverage hardware TCP acceleration present in silicon, assisting lower spec processors with NDI transport.[28] NDI 4.0 shipped in September 2019 to coincide with the IBC exhibition.

Version Released Features/Enhancements
1.0 April 2016 Initial Release
2.0 Sept 2016 Cross subnet support via Access Manager, ARM encoding support, iOS SDK
3.0 July 2017 NDI-HX support, Option for multicast transport with FEC, PTZ support.
3.5 June 2018 Defaults to Unicast UDP transport method with FEC.
3.6 July 2018 Packet level bonding improvements.
3.7 Sept 2018 Discovery improvements.
3.8 Nov 2018 Performance Enhancements, NDI Embedded SDK for FPGA development.
4.0 Sept 2019 Multi-TCP Transport mode, HDR 16-bit support, Native synchronised recording, Discovery Server, NDI HX2[29]
4.1 Nov 2019 Performance improvements.

Use in Wifi and Wide Area Networks[edit]

NDI was designed to work on good quality gigabit local area networks using TCP and Bonjour (mDNS) technologies. In order to work across subnets which don't pass mDNS, NDI supports a mechanism known as NDI Access which allows manual entry of the IP address of machines on other subnet which may be running NDI sources.

Some NDI adopters have run the protocol across medium distance Fibre connections up to 15 km, although NDI's use of the TCP protocol make it less suitable for longer distance, high latency connections due to factors such as Bandwidth Delay Product and TCP packet loss recovery.[30] The NDI 3.5 libraries now default to a UDP transfer method with forward error correction that removes the latency requirements that are inherent with TCP connections. NDI|HX uses a lower data rate making it easier to use in bandwidth limited connections.[24]

To provide an extension of NDI to wide area networks, tools have been developed including the Sienna Cloud for NDI[31] protocol which uses node gateways in each LAN to bridge NDI sources across continents.[32] In July 2017, USSSA (United States Speciality Sports Association) successfully made the first truly wide area network NDI sports coverage using Cloud for NDI with 5 cameras backhauled across the public internet from a stadium, to 200 miles away into an NDI based production gallery.[33]

NDI is also used within cloud based production systems[34], with its compressed video and non-reliance on multicast making it a perfect fit for cloud based data centres like AWS and Azure. in 2019 Sienna demonstrated a complete cloud based production workflow using NDI, including remote contribution using Sienna. Cloud or SRT, cloud-based processing, including VisionMixing, Graphics and Audio manipulation, leading to final publishing via CDNs like YouTube Live.

CPU architecture support[edit]

NDI, when running on x86 architectures, requires CPU's that include the SSSE3 (Supplemental Streaming SIMD Extensions 3) instruction set. Most Intel CPU designs starting in 2006 have this instruction set, AMD added support starting in 2011.

While not a requirement, NDI will take advantage of AVX (Advanced Vector Extensions) and AVX2 instructions sets for additional performance. While NDI itself doesn't require these instructions, applications might.[citation needed]

NDI can run on 32-bit or 64-bit CPU architectures, although performance is increased when using 64-bit, and 32-bit support may not be available in future versions of the macOS and iOS NDI Libraries.

NDI is available for ARM based systems with encode support, but generally not video decode. An exception to this is Sienna NDI Monitor[35] for iOS which does provide NDI Decoding on ARM.

NDI HX Devices are typically send-only and based on proprietary platforms with hardware H.264 encoder chips. Examples are NDI HX PTZ Cameras, and the NDI Connect Spark SDI to NDI HX box. The NDI Stream itself is formed at the stream consumption end via the platform specific NDI HX driver, presenting the same uncompressed data to the application software consuming the NDI source regardless of original HX source hardware and compression. NDI HX2 is a purer implementation of h.264 in NDI packets, and will allow software and hardware systems to embed h.264 essence into NDI streams using the NDI Embedded SDK.

NewTek, Birddog and Magewell have all developed FPGA encoding implementations of NDI. In April 2018 NewTek announced the Connect Spark Pro which will be their first FPGA based NDI product. Birddog and Magewell also offer FPGA based decode of full NDI streams (not NDI HX). Newtek offers a licensable FPGA reference design for 3rd party NDI Encoders as part of their NDI Embedded SDK

With NDI 4.0 NewTek announced the addition of a 'Multi-TCP' transport mode. This takes advantage of the hardware TCP acceleration in silicon which helps lower spec processors handle heavy network load, in contrast to UDP which does not benefit from the hardware acceleration.[36]

Metadata and Extensions to the NDI Specification[edit]

NDI Supports arbitrary metadata as XML blocks, embedded in video and audio frames, as well as stand-alone metadata frames. The content of these metadata blocks form 3 families.

  • Internal Metadata used by NDI invisibly. These messages are used for connectivity and some other fundamental tasks like Tally. They are typically invisible to NDI clients.
  • Defined Public Metadata. These messages include things like the NDI PTZ Protocol. They are defined by NewTek as part of the NDI SDK.
  • 3rd Party Metadata schemes. These messages are implemented in the same way as Defined NDI Metadata but the content is based on 3rd party designs. Over time these 3rd party metadata schemes may become widely used and perhaps adopted as officially supported.

There are 6 published examples of 3rd Party NDI Metadata Schemes:

  • In August 2018, Gallery Sienna published a proposal[37] for the carriage of C708 Closed Caption metadata via NDI. It involves capture of the appropriate line of SDI VANC data, and text based encoding to be compatible with XML. This is carried as video frame-based attached metadata. It can be passed to an equivalently compatible decoder for reconversion to SDI along with intact C708 captions. Gallery Sienna implemented this standard into their Sienna NDI SourceMulti product[38], and the proposal is also being adopted by other NDI product developers.
  • In February 2019 Gallery Sienna published a proposal[39] for the carriage of MIDI messages via NDI. This uses real time stand alone NDI Metadata frames to carry any sort of Midi message including Midi Machine Control, and controller data. NDI Midi is used within Sienna's NDI Processing Engine[40] as a generic control protocol between modules and the outside world.
  • In April 2019 Gallery Sienna published a proposal[41] for the use of the industry standard iXML Metadata protocol within NDI. iXML allows for unambiguous description of stream contents, and linkage between streams.
  • In April 2019 Gallery Sienna published a proposal[42] for the carriage of PTPv2 timing and GMID within NDI. Sienna also implemented this as a Precision_Time_Protocol converter which captures this data and feeds it as NDI into multiple NDI systems wanting to share this information, such as software to interact with AES67 devices.
  • In September 2019 Gallery Sienna published a proposal[43] for the definition and carriage of sub-region stream definition within NDI. This can be used for definition of video wall subcuts of a stream, or for applications like AI analysis of video where a system is following a face in a stream, or following a football - with a pan scan subcut of the stream. This is also used to deliver sub-stream definitions for social media publishing with non standard aspect ratios - for example, portrait or square video for phones.
  • In November 2019 Gallery Sienna published a proposal[44] for the definition of the HDR Metadata description profile for HDR NDI Streams

Other non-published NDI Metadata schemes are also in use, for things like triggering CG animations.

References[edit]

  1. ^ "NewTek NDI SDK". Newtek.com.
  2. ^ "Home". Sienna-tv.com.
  3. ^ a b "bird-dog". Bird-dog.
  4. ^ Coalition, ProVideo (8 September 2015). "NewTek announces NDI, an open protocol for IP production workflow by Allan Tépper".
  5. ^ "NDI Encoding/Decoding". NewTek Knowledge Base.
  6. ^ "NDI Network Bandwidth". NewTek Knowledge Base.
  7. ^ "NewTek Announces NDI – Open Protocol for IP Production Workflow - Technical Standards". Content-technology.com.
  8. ^ a b "Num" (PDF). Smpte.org. Retrieved 2017-05-23.
  9. ^ "ASPEN Community - Home". aspen-community.com.
  10. ^ "Sony launches Networked Media Interface in collaboration with leading broadcast industry organisations : Press : United Kingdom : Sony Professional". Sony.co.uk.
  11. ^ "SpeedHQ - MultimediaWiki". wiki.multimedia.cx.
  12. ^ "NMI Core - Nextera Video Inc". Nexteravideo.com.
  13. ^ "What are the Networked Media Open Specifications?". Nmos.tv.
  14. ^ "NewTek NDI SDK". NewTek.
  15. ^ "TICO Alliance - Technology". www.tico-alliance.org.
  16. ^ "SMPTE RDD 35 - TICO Lightweight Codec Used in IP Networked or in SDI Infrastructures - Engineering360". standards.globalspec.com.
  17. ^ "NewTek announces NDI, an open protocol for IP production workflow by Allan Tépper". Provideocoalition.com. 8 September 2015.
  18. ^ Kerschbaumer, Ken; Monday, Editorial Director; 19, September; Story, 2016-9:34 am Print This. "NewTek's IP Series Video-Production System Targets New Markets". Sports Video Group.
  19. ^ "First 3rd Party NDI Tools Available !". forums.newtek.com.
  20. ^ "NDICam". Sienna-tv.com.
  21. ^ "NDI AirSend Updater-NewTek". pages.newtek.com.
  22. ^ Bridge, The Broadcast. "StudioCoast vMix Adopts NewTek NDI Standard - The Broadcast Bridge - Connecting IT to Broadcast". Thebroadcastbridge.com.
  23. ^ "XSplit Broadcaster Adopts NewTek NDI for IP Production Workflow". Marketwired.com. 2016-04-12. Retrieved 2017-05-23.
  24. ^ a b "NewTek NDI® Version 3 Offers the Only End-to-End IP Video Solution for Product Manufacturers". NewTek.
  25. ^ "Make collaboration the heart of your live streams, podcasts, and videos with Skype for Content Creators". Skype Blogs. 6 April 2018.
  26. ^ Director, Ken Kerschbaumer, Editorial. "EVS X-One Production System Takes Software-Based Tech to New Level". Sports Video Group.
  27. ^ "Avid Media Composer Integrates NewTek NDI®".
  28. ^ "NewTek's Andrew Cross Discusses Acquisition, NAB Show Plans".
  29. ^ "NewTek's Andrew Cross Discusses Acquisition, NAB Show Plans".
  30. ^ Gatarski, Richard (9 May 2016). "Documentary: Stockholm Kista cross production using NDI over fiber". Weststreamu.se.
  31. ^ "NDIRelay". Sienna-tv.com.
  32. ^ "Sienna Takes NDI into the Cloud - NewTek Studio". blog.newtek.com.
  33. ^ "ndicloudblog". www.sienna-tv.com.
  34. ^ http://www.sienna-tv.com/ndi/infrastructure.html
  35. ^ "Ndimonios".
  36. ^ "NewTek's Andrew Cross Discusses Acquisition, NAB Show Plans".
  37. ^ "Ndiclosedcaptions".
  38. ^ "NDI Source Multi".
  39. ^ "Midioverndi".
  40. ^ "NDIProcessor".
  41. ^ "Ndiixml".
  42. ^ "Ndiptpsync".
  43. ^ "NDI Regions".
  44. ^ "NDI HDR Description".