Opus (audio format)

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by PieterDeBruijn (talk | contribs) at 11:14, 12 January 2013 (→‎VoIP software: correct wording). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Opus
Filename extension
.opus
Internet media type
audio/ogg,[1] audio/opus[2]
Developed byIETF codec working group
Initial releaseSeptember 11, 2012 (2012-09-11)
Type of formatAudio
Contained byOgg
Extended fromSILK, CELT
StandardRFC 6716
Free format?Yes
Websiteopus-codec.org
libopus
Developer(s)Xiph.Org Foundation
Initial releaseAugust 26, 2012 (2012-08-26)
Stable release
1.0.2 / December 6, 2012; 11 years ago (2012-12-06)
Preview release
1.1-alpha / December 21, 2012; 11 years ago (2012-12-21)
Written inC89
PlatformCross-platform
TypeAudio codec, reference implementation
License3-clause BSD license
WebsiteOpus codec downloads

In computing, Opus is a lossy audio compression format developed by the Internet Engineering Task Force (IETF) and made especially suitable for interactive real-time applications over the Internet.[3] As an open format standardized through RFC 6716, a reference implementation is provided under the 3-clause BSD license. All known software patents which cover Opus are licensed under royalty-free terms.[4]

Opus incorporates technology from two codecs: the speech-oriented SILK and the low-latency CELT.[3] Opus can be adjusted seamlessly between high and low bitrates, and internally, it transitions between a linear prediction codec at lower bitrates and a transform codec at higher bitrates (as well as a hybrid for a short overlap). Opus has a very low algorithmic delay (22.5 ms by default), which is a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay, down to 5 ms. Its delay is good compared to well over 100 ms for popular music formats such as MP3, Ogg Vorbis and HE-AAC; yet Opus performs very competitively with these codecs in terms of quality per bitrate.[5] Unlike these codecs, Opus does not require the definition of large codebooks for each individual file, making it preferable also for short clips of audio.[6]

Features

Possible bitrate and latency combinations compared with other codecs

Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and various sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, where the entire hearing range of the human auditory system can be reproduced). An Opus stream can support up to 255 audio channels, allowing channel coupling between two at a time using mid-side coding.

The inherently low delay in Opus (22.5 ms by default) makes it possible to be used in the same real-time applications required by telephony, Voice over IP and videoconferencing. For high quality audio, the ability to maintain low latency at higher bitrates is made possible by research done by the Xiph.Org Foundation on the CELT codec. During any Opus stream (live or in a file), the bitrate, bandwidth, and delay can be changed quickly and seamlessly without introducing any distortion or discontinuity in the audio.

As an open standard, the algorithms are openly documented, and a reference implementation (including the source code) is published. Broadcom and the Xiph.Org Foundation own software patents on some of the CELT algorithms, and Skype Technologies S.A./Microsoft own some on the SILK algorithms; but each has pledged to make them available royalty-free for use with Opus once the codec was accepted as an IETF standard. They also reserve the right to make use of their patents to defend against infringement suits of third parties. The applicability of non royalty-free patent claims from Qualcomm and Huawei to Opus is disputed.[7]

Spectrogram of an Opus-encoded music recording at different average bitrates (~32 to ~160 kbit/s) clearly show the lowpass behavior of the encoder and the comparatively good preservation of the band energy with CELT (compare original, Vorbis, MP3, AAC).

The Opus codec is based on the low-latency CELT codec and the speech-oriented SILK codec (both of which have been heavily modified, rendering them incompatible with their original formats). The transform layer (CELT) is based on the modified discrete cosine transform (MDCT) with approaches from CELP (codebooks for excitation, although in the frequency domain). CELT was modified and among other things, support for 20 ms frames was added. The SILK layer that specializes in speech signals is based on linear predictive coding (LPC). SILK was modified and among other things, support for 10 ms frames was added. To minimize packet overhead, SILK has support for a longer delay of 60 ms (versus CELT's 20 ms). The shared range encoding of both parts of a hybrid stream was taken from CELT.

The codec has three different modes, two being for pure speech signals, and a third for general audio (including music). One of the speech modes is capable of reproducing the full spectrum of the human hearing range. In this mode, CELT is used for the upper part of the frequency range (from 8 kHz upwards), and SILK is used for the lower part. For low bitrates (below about 30 kbit/s), the upper frequencies can be cut off and the CELT layer left out. For audio at higher bitrates, the SILK layer that specializes in speech signals is left out, and the non-specialized CELT layer is used.

The reference implementation is written in C and compiles on hardware architectures with or without floating-point unit.

Opus data can be encapsulated in Ogg containers. The content of such Ogg Opus streams should be specified as audio/ogg; codecs=opus and for Ogg Opus files the .opus filename extension is recommended.[1] Support for encapsulation of Opus in Matroska containers is being worked on.[8]

History

See also: CELT § History, and SILK § History

Opus was proposed for the standardization of a new audio codec at the IETF, which was eventually accepted and granted the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A. (now Microsoft). Its main developers are Jean-Marc Valin (Xiph.Org, Octasic, Mozilla Corporation), Koen Vos (Skype), and Timothy B. Terriberry (Xiph.Org, Mozilla Corporation). Among others, Juin-Hwey (Raymond) Chen (Broadcom), Gregory Maxwell (Xiph.Org, Wikimedia), and Christopher Montgomery (Xiph.Org) were also involved.

The development of the CELT part of the codec goes back to thoughts on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.

The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.

In March 2009, Skype suggested the development and standardization of a wideband audio codec within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group.[9] Representatives of several companies which were taking part in the standardization of patent-encumbered competing formats stated objections against the start of the standardization process for a royalty-free codec: representatives of Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs (department of France Télécom), which were involved in the creation of G.718. The working group finally formed in February 2010 and even the corresponding Study Group 16 from the ITU-T pledged to support its work.

In July 2010, a prototype of a hybrid format was presented that combined the two proposed codec candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010.[10][11] At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes.[12] Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus.[13] In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012.[14] On July 2, 2012, Opus was approved by the IETF for standardization.[15] The reference software entered release candidate state on August 8.[16] The final specification was released as RFC 6716 on September 10, 2012.[17][7] and versions 1.0 and 1.0.1 of the reference implementation were released the day after. Meanwhile, there has been development work on an experimental branch of the reference encoder that, among other things, makes much broader use of variable bitrate mechanisms of the format and is reported to produce significantly higher audio quality, especially on very tonal samples.

Quality comparison and low latency performance

Comparison of coding efficiency between Opus and other popular codecs

Opus has been shown to have excellent quality,[5] and at higher bit rates, it turns out to be competitive with codecs with much higher delay, such as HE-AAC and Vorbis.[18]

In listening tests the codec shows superior quality compared to HE-AAC codecs, which have so far been dominant in that area because of their use of the proprietary spectral band replication (SBR) technology.[19][20]

Opus has very low algorithmic delay,[3] a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Total algorithmic delay for a codec is the sum of delays that must be incurred in the encoder and the decoder of a live audio stream regardless of processing speed and transmission speed, such as buffering audio samples into blocks or frames, allowing for window overlap and possibly allowing for noise-shaping look-ahead in a decoder and any other forms of look-ahead, or for an MP3 encoder, the use of bit reservoir.[21]

Total one-way latency below 150 ms is the preferred target of most VoIP systems,[22] to enable natural conversation with turn-taking little affected by delay. Musicians typically feel in-time with up to around 30 ms audio latency,[23] roughly in accord with the fusion time of the Haas effect, though matching playback delay of each user's own instrument to the round-trip latency can also help.[24] It is suggested for lip sync that around 45-100 ms audio latency may be acceptable.[25]

Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay (5.0 ms minimum).[26] While the default Opus frame is 20 ms long, a further 2.5 ms lookahead is required for window overlap in the CELT layer and noise shaping in the SILK layer, typically giving a delay of 22.5 ms. The SILK layer's minimum is 10 ms frames (12.5 ms delay), while the CELT layer's minimum is 2.5 ms (5.0 ms delay).[27]

Support

The format and algorithms are openly documented and the reference implementation is published as free software. The reference implementation (Opus Audio Tools, opus-tools), consisting of separate encoders and decoders, is published under the terms of a BSD-like license. It is written in C programming language and can be compiled for hardware architectures with or without floating point unit. The accompanying diagnostic tool opusinfo reports detailed technical information about Opus files, including information on the standard compliance of the bitstream format. It is based on ogginfo from the vorbis-tools and therefore, unlike the encoder and decoder, available under the terms of version 2 of the GPL.

VoIP software

  • The voice-chat software Mumble supports Opus as its main codec.[28][29]
  • SIP softphones Phoner and PhonerLite support Opus (support was originally added when Opus was still in the early draft phase)
  • The SIP and IAX2 client SFLphone is being fitted with Opus support.[30]
  • Integration of Opus into the Skype client is finished, although no version with Opus support has yet been published.
  • TrueConf video conferencing solutions support Opus.[31]
  • Opus support is planned for Jitsi 2.0, together with VP8 video[32][33]
  • Empathy may use any format supported in GStreamer, including Opus.
  • Line2 has replaced their current codec with Opus. Their iOS app will be the first to be released with the Opus. The Android app will follow later.
  • CSipSimple supports Opus, Codec2, G.726 and G.722.1 with an additional plug-in.
  • The voice-chat software TeamSpeak 3 supports Opus for voice and music in pre-release server 3.0.7-pre2 and beta client version 3.0.10 [34][35]

Web frameworks and browsers

  • Opus support is mandatory for WebRTC implementations.[36]
  • Mozilla supports Opus beginning with version 15 of Firefox and Thunderbird.[37]
  • Depending on the backend in use, Opera supports inline playback of embedded Opus files.[38] Official support for Opus and WebRTC are on the development roadmap.[39]
  • Chromium and Google Chrome will have audio support as of version 25.

Streaming audio

Operating systems and desktop multimedia frameworks

  • In Debian GNU/Linux the Opus development tools and supporting libraries can be installed from the preconfigured repositories in the next stable version ("wheezy") that is expected to be released in early 2013.[43]
  • For Microsoft Windows, there are DirectShow filters supporting Opus, including DC-Bass Source Mod and the LAV Filters.[44]
  • In GStreamer the integration of Opus support is complete.[45]
  • FFmpeg supports decoding and encoding Opus via the external library libopus.

Hardware support

  • Support in Rockbox is available in the developer version.[46] This means hardware support for a series of portable media players (products from the iPod series by Apple and iriver and Archos devices) and with "Rockbox as an Application" (RaaA)[47] also on Android devices.

Player software

  • VLC media player supports Opus since version 2.0.4[48]
  • AIMP supports Opus natively as of version 3.20 build 1125 beta 1.[49]
  • foobar2000 supports the format natively as of v1.1.14 beta 1.[50]
  • Mpxplay supports Opus (using a decoder DLL) as of v1.60 alpha 2

Other software

See also

References

  1. ^ a b "Ogg Encapsulation for the Opus Audio Codec". IETF. 2012-07-16. Retrieved 2012-09-12.
  2. ^ Network Working Group (July 4, 2011). "RTP Payload Format and File Storage Format for Opus Speech and Audio Codec". Opus codec. IETF. Retrieved October 26, 2011.
  3. ^ a b c "Opus Codec". Opus (Home page). Xiph.org Foundation. Retrieved July 31, 2012.
  4. ^ http://arstechnica.com/gadgets/2012/09/newly-standardized-opus-audio-codec-fills-every-role-from-online-chat-to-music/
  5. ^ a b Raymond Chen et al. Opus Testing. IETF 80
  6. ^ Terriberry, Timothy (July 19, 2012). "Firefox Beta 15 supports the new Opus audio format". Mozilla Hacks. Mozilla Foundation. Retrieved July 31, 2012.
  7. ^ a b "It's Opus, it rocks and now it's an audio codec standard!". Mozilla Hacks. Retrieved 2012-09-12.
  8. ^ "MatroskaOpus". XiphWiki. 2012-07-05. Retrieved 2012-09-12.
  9. ^ "IETF working towards royalty-free audio codec". H-online.com. 2009-11-13. Retrieved 2012-09-12.
  10. ^ Vos, Koen (2010-10-15). "Harmony became Opus". ietfcodec Discussion Archive (Mailing list). Retrieved 2011-06-19. {{cite mailing list}}: External link in |mailinglist= (help); Unknown parameter |mailinglist= ignored (|mailing-list= suggested) (help)
  11. ^ "[codec] Harmony became Opus". Ietf.org. 2010-10-15. Retrieved 2012-09-12.
  12. ^ "IETF Opus codec now ready for testing". Hydrogenaudio.org. Retrieved 2012-09-12.
  13. ^ Valin, Jean-Marc (2011-08-01). "IETF update, Mozilla". LiveJournal.com. Retrieved 2012-10-05.
  14. ^ "Opus Codec". Opus Codec. Retrieved 2012-09-12.
  15. ^ "Opus approved by the IETF". Jmspeex.livejournal.com. 2012-07-03. Retrieved 2012-09-12.
  16. ^ "[opus] Release candidates for 1.0.0 and 1.0.1 are available". Lists.xiph.org. Retrieved 2012-10-05.
  17. ^ "RFC 6716 on Definition of the Opus Audio Codec". Ietf.org. Retrieved 2012-09-12.
  18. ^ Maxwell, Gregory (2011). "64kbit/sec stereo multiformat listening test — unofficial results page". Xiph.Org Foundation. Retrieved 2011-06-19.
  19. ^ Next-Gen Low-Latency Open Codec Beats HE-AAC, Slashdot-Meldung vom 14. April 2011
  20. ^ Valin, Jean-Marc; Vos, Koen; Skoglund, Jan (2011-10-24). Hoene, Christian (ed.). "Summary of Opus listening test results". Internet Engineering Task Force. Retrieved 2012-01-16.
  21. ^ Lutzky, Manfred (2004.). "A guideline to audio codec delay" (PDF). In AES 116th convention, Berlin, Germany: 8–11. Retrieved 30 October 2012. {{cite journal}}: Check date values in: |year= (help); Unknown parameter |coauthors= ignored (|author= suggested) (help)
  22. ^ Lironi, F. (2005). "Multi RAB-based multimedia services over GERAN mobile networks". Vehicular Technology Conference, VTC-2005-Fall. IEEE 62nd. 3: 1662–1666. Retrieved 30 October 2012. {{cite journal}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
  23. ^ Lago, Nelson Posse (2004). "The quest for low latency" (PDF). In Proceedings of the International Computer Music Conference: 33–36. Retrieved 30 October 2012. {{cite journal}}: Unknown parameter |coauthors= ignored (|author= suggested) (help); Unknown parameter |month= ignored (help)
  24. ^ Carôt, Alexander (2010). "Low Latency Audio Streaming for Internet-Based Musical Interaction." in Streaming Media Architectures: Techniques and Applications: Recent Advances. IGI Global. pp. 362–383. ISBN 1616928336, 9781616928339. {{cite book}}: Check |isbn= value: invalid character (help)
  25. ^ YE, NONG, VAN CHEN, and TONI FARLEY (2003). "Qos Requirements Of Multimedia Data On Computer Networks." Proceedings of the Second International Conference on Active Media Technology, Chongqing, PR China, 29-31 May 2003. World Scientific Publishing Company Incorporated. pp. 183–189. ISBN 9812383433, 9789812383433. {{cite book}}: Check |isbn= value: invalid character (help)CS1 maint: multiple names: authors list (link)
  26. ^ Montgomery, Christopher. "A quick showcase of the bleeding edge... CELT 0.10.0 @ constant PEAQ value, varying latency". CELT v0.10 (latest prior to Opus integration). xiph.org. Retrieved 30 October 2012.
  27. ^ Valin, Jean-Marc; et al. "Opus Codec Overview". IETF RFC 6716. IETF. Retrieved 30 October 2012. {{cite web}}: Explicit use of et al. in: |first= (help)
  28. ^ "Kommentare zu: Was ich so höre ... Liste freier Musik". Natenom.name. 2010-10-14. Retrieved 2012-09-12.
  29. ^ "1.2.4 - Mumble". Mumble. Retrieved 2012-09-12.
  30. ^ "SFLphone — Task #14602: [Codec] Implement opus". Savoir-faire Linux. 2012-08-13. Retrieved 2012-09-12.
  31. ^ TrueConf supports Opus
  32. ^ "(SIP Communicator) | GSOC / Opus and CELT support in". Jitsi. Retrieved 2012-09-12.
  33. ^ "(SIP Communicator) | Development / Roadmap". Jitsi. Retrieved 2012-09-12.
  34. ^ "[Beta Channel] TeamSpeak 3 Client 3.0.10". TeamSpeak Forums. Retrieved 2012-12-10.
  35. ^ "[PreRelease] TeamSpeak 3 Server 3.0.7 PreRelease 2". TeamSpeak Forums. Retrieved 2012-12-10.
  36. ^ "Zwei Audio-Codecs für Echtzeit-Kommunikation im Browser". Heise.de. Retrieved 2012-09-12.
  37. ^ "Media formats supported by the HTML audio and video elements". Developer.mozilla.org. 2012-09-03. Retrieved 2012-09-12.
  38. ^ "Why no official mention of Opus support?". My.opera.com. 2012-07-19. Retrieved 2012-09-12.
  39. ^ Shankland, Stephen (2012-08-17). "How corporate bickering hobbled better Web audio". CNET News. Retrieved 2012-09-12.
  40. ^ "Icecast 2.4 beta release". Lists.xiph.org. Retrieved 2012-09-12.
  41. ^ Absolute Radio, Listen Labs. "Opus Streaming Trial". Absolute Radio. Retrieved 30 October 2012.
  42. ^ "krad-radio/krad_radio · GitHub". Github.com. 2012-07-26. Retrieved 2012-10-05.
  43. ^ Debian Webmaster, webmaster@debian.org. "Details of package opus-tools in wheezy". Packages.debian.org. Retrieved 2012-10-05.
  44. ^ "lavfilters — Open-Source DirectShow Media Splitter and Decoders". Google Project Hosting. Retrieved 2012-10-05.
  45. ^ "GStreamer: news". Gstreamer.freedesktop.org. Retrieved 2012-10-05.
  46. ^ Rockbox Contributors. "Rockbox code repository". Rockbox. Retrieved 2012-10-14. {{cite web}}: |author= has generic name (help)
  47. ^ "RockboxAsAnApplication2010 < Main < Wiki". Rockbox.org. Retrieved 2012-10-05.
  48. ^ "VLC 2.0.4 Twoflower". VideoLAN. Retrieved 2012-10-19.
  49. ^ "AIMP: Home Page". Aimp.ru. Retrieved 2012-10-05.
  50. ^ "foobar2000 v1.1.14 beta". Hydrogenaudio Forums. Retrieved 2012-10-05.
  51. ^ "[Tieline Integrates OPUS into Report-IT and IP codecs". Radio Magazine. Retrieved 2012-12-18.

External links

Wikimedia Commons has media related to Opus.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Opus_(audio_format)&oldid=532682645"