Opus (audio codec)

From Wikipedia, the free encyclopedia
  (Redirected from Opus (codec))
Jump to: navigation, search
Opus
Opus logo2.svg
Filename extension .opus
Internet media type audio/ogg,[1] audio/opus[2]
Developed by IETF codec working group
Initial release September 11, 2012 (2012-09-11)
Type of format Audio
Contained by Ogg, Matroska
Extended from SILK, CELT
Standard(s) RFC 6716
Open format? Yes
Website opus-codec.org
libopus
Developer(s) Xiph.Org Foundation
Initial release August 26, 2012 (2012-08-26)
Stable release 1.1 / December 5, 2013; 4 months ago (2013-12-05)
Preview release 1.1-rc / November 26, 2013; 4 months ago (2013-11-26)
Written in C89
Platform Cross-platform
Type Audio codec, reference implementation
License 3-clause BSD license
Website Opus codec downloads

Opus is a lossy audio compression format developed by the Internet Engineering Task Force (IETF) that is particularly suitable for interactive real-time applications over the Internet.[3] As an open format standardized through RFC 6716, a reference implementation is provided under the 3-clause BSD license. All known software patents which cover Opus are licensed under royalty-free terms.[4]

Opus incorporates technology from two codecs: the speech-oriented SILK and the low-latency CELT.[3] Opus can be adjusted seamlessly between high and low bitrates, and internally, it transitions between a linear prediction codec at lower bitrates and a transform codec at higher bitrates (as well as a hybrid for a short overlap). Opus has a very low algorithmic delay (22.5 ms by default), which is a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay, down to 5 ms. Its delay is very low compared to well over 100 ms for popular music formats such as MP3, Ogg Vorbis and HE-AAC; yet Opus performs very competitively with these codecs in terms of quality per bitrate.[5] Unlike Ogg Vorbis, Opus does not require the definition of large codebooks for each individual file, making it preferable to Vorbis for short clips of audio.[6]

Features[edit]

Possible bitrate and latency combinations compared with other codecs

Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and certain sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, where the entire hearing range of the human auditory system can be reproduced). An Opus stream can support up to 255 audio channels, and it allows channel coupling between channels in groups of two using mid-side coding.

The inherently low delay in Opus (22.5 ms by default) makes it possible to be used in the same real-time applications required by telephony, Voice over IP and videoconferencing. For high quality audio, the ability to maintain low latency at higher bitrates is made possible by research done by the Xiph.Org Foundation on the CELT codec. During any Opus stream (live or in a file), the bitrate, bandwidth, and delay can be changed quickly and seamlessly without introducing any distortion or discontinuity in the audio.

As an open standard, the algorithms are openly documented, and a reference implementation (including the source code) is published. Broadcom and the Xiph.Org Foundation own software patents on some of the CELT algorithms, and Skype Technologies S.A./Microsoft own some on the SILK algorithms; but each has pledged to make them available royalty-free for use with Opus once the codec was accepted as an IETF standard. They also reserve the right to make use of their patents to defend against infringement suits of third parties. The applicability of non royalty-free patent claims from Qualcomm and Huawei to Opus is disputed.[7]

Spectrogram of an Opus-encoded music recording at different average bitrates (~32 to ~160 kbit/s) clearly show the lowpass behavior of the encoder and the comparatively good preservation of the band energy with CELT (compare original, Vorbis, MP3, AAC).

The Opus codec is based on the low-latency CELT codec and the speech-oriented SILK codec (both of which have been heavily modified, rendering them incompatible with their original formats). The transform layer (CELT) is based on the modified discrete cosine transform (MDCT) with approaches from CELP (codebooks for excitation, although in the frequency domain). CELT was modified and among other things, support for 20 ms frames was added. The SILK layer that specializes in speech signals is based on linear predictive coding (LPC) and an optional Long-Term Prediction filter. SILK was modified and among other things, support for 10 ms frames was added. To minimize packet overhead, SILK has support for a longer delay of 60 ms (versus CELT's 20 ms). The shared range encoding of both parts of a hybrid stream was taken from CELT.

The codec has three different modes, two being for pure speech signals, and a third for general audio (including music and speech). One of the speech modes is capable of reproducing the full spectrum of the human hearing range. In this mode, CELT is used for the upper part of the frequency range (from 8 kHz upwards), and SILK is used for the lower part. For low bitrates (below about 30 kbit/s), the upper frequencies can be cut off and the CELT layer left out. For audio at higher bitrates, the SILK layer that specializes in speech signals is left out, and the non-specialized CELT layer is used.

The reference implementation is written in C and compiles on hardware architectures with or without a floating-point unit.

Opus data can be encapsulated in Ogg containers. The content of such Ogg Opus streams should be specified as audio/ogg; codecs=opus and for Ogg Opus files the .opus filename extension is recommended.[1] Support for encapsulation of Opus in Matroska containers has been completed.[8]

History[edit]

Opus was proposed for the standardization of a new audio codec at the IETF, which was eventually accepted and granted by the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A. (now Microsoft). Its main developers are Jean-Marc Valin (Xiph.Org, Octasic, Mozilla Corporation), Koen Vos (Skype), and Timothy B. Terriberry (Xiph.Org, Mozilla Corporation). Among others, Juin-Hwey (Raymond) Chen (Broadcom), Gregory Maxwell (Xiph.Org, Wikimedia), and Christopher Montgomery (Xiph.Org) were also involved.

The development of the CELT part of the codec goes back to thoughts on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.

The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.

In March 2009, Skype suggested the development and standardization of a wideband audio codec within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group.[9] Representatives of several companies which were taking part in the standardization of patent-encumbered competing formats stated objections against the start of the standardization process for a royalty-free codec: representatives of Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs (department of France Télécom), which were involved in the creation of G.718. The working group finally formed in February 2010, and even the corresponding Study Group 16 from the ITU-T pledged to support its work.

In July 2010, a prototype of a hybrid format was presented that combined the two proposed codec candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010.[10][11] At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes.[12] Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus.[13] In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012.[14] On July 2, 2012, Opus was approved by the IETF for standardization.[15] The reference software entered release candidate state on August 8.[16] The final specification was released as RFC 6716 on September 10, 2012.[7][17] and versions 1.0 and 1.0.1 of the reference implementation were released the day after.

On July 11, 2013, the first beta of version 1.1 was released incorporating significant encoder side improvements for quality and efficiency, and significant speed improvements for both encoding and decoding detailed in the release information. Surround sound encoding has better channel bitrate allocation, improved Low-Frequency Effects quality and efficiency, and use of cross-channel masking. Tonality estimation provides a bitrate boost to highly tonal sounds like harpsichord, significantly improving quality in Opus's weakest samples from previous public 64kbps tests in which it had otherwise performed well. Automated speech/music detection is enabled to improve quality. Other improvements include stereo bitrate saving when channels are closely correlated, dynamic allocation to improve quality for transients and high-energy bands adjacent to low-energy bands, DC rejection below 3 Hz to improve bass analysis. True unconstrained VBR improves quality consistency for same target bitrate, while Temporal VBR is a serendipitous discovery that improves quality on a number of samples.

Also on July 11, 2013, stable version 1.0.3 was introduced implementing a backport of the new 1.1 surround API mentioned in beta 1.1 above as well as some minor bug fixes.

Opus 1.1 was officially released on December 5, 2013.

Quality comparison and low latency performance[edit]

Comparison of coding efficiency between Opus and other popular codecs

Opus has been shown to have excellent quality,[5] and at higher bit rates, it turns out to be competitive with codecs with much higher delay, such as HE-AAC and Vorbis.[18]

In listening tests at around 64 kbit/s Opus shows superior quality compared to HE-AAC codecs, which have so far been dominant in that area because of their use of the proprietary spectral band replication (SBR) technology.[19][20]

Opus has very low algorithmic delay,[3] a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Total algorithmic delay for a codec is the sum of delays that must be incurred in the encoder and the decoder of a live audio stream regardless of processing speed and transmission speed, such as buffering audio samples into blocks or frames, allowing for window overlap and possibly allowing for noise-shaping look-ahead in a decoder and any other forms of look-ahead, or for an MP3 encoder, the use of bit reservoir.[21]

Total one-way latency below 150 ms is the preferred target of most VoIP systems,[22] to enable natural conversation with turn-taking little affected by delay. Musicians typically feel in-time with up to around 30 ms audio latency,[23] roughly in accord with the fusion time of the Haas effect, though matching playback delay of each user's own instrument to the round-trip latency can also help.[24] It is suggested for lip sync that around 45-100 ms audio latency may be acceptable.[25]

Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay (5.0 ms minimum).[26] While the default Opus frame is 20 ms long, a further 2.5 ms lookahead is required for window overlap in the CELT layer and noise shaping in the SILK layer, typically giving a delay of 22.5 ms. The SILK layer's minimum is 10 ms frames (12.5 ms delay), while the CELT layer's minimum is 2.5 ms (5.0 ms delay).[27]

Support[edit]

The format and algorithms are openly documented and the reference implementation is published as free software. Xiph's reference implementation is called opus-tools, and it consists of a separate encoder and decoder; it is published under the terms of a BSD-like license. It is written in C and can be compiled for hardware architectures with or without a floating point unit. The accompanying diagnostic tool opusinfo reports detailed technical information about Opus files, including information on the standard compliance of the bitstream format. It is based on ogginfo from the vorbis-tools and therefore — unlike the encoder and decoder — is available under the terms of version 2 of the GPL.

VoIP software[edit]

  • Asterisk has no builtin support for legal reasons,[28] but 3rd party patch that adds Opus support is available for download.[29]
  • The voice-chat software Mumble in version 1.2.4 supports Opus as its main codec.[30][31][32][33]
  • SIP softphones Phoner and PhonerLite support Opus (support was originally added when Opus was still in the early draft phase)
  • The SIP and IAX2 client SFLphone is being fitted with Opus support.[34]
  • TrueConf video conferencing solutions support Opus.[35]
  • Jitsi supports Opus together with VP8 video[36][37] in v2.0 release.[38]
  • Empathy may use any format supported in GStreamer, including Opus.
  • Line2 has replaced their current codec with Opus. Their iOS app will be the first to be released with Opus. The Android app will follow later.[39]
  • CSipSimple supports Opus with an additional plug-in.
  • The voice-chat software TeamSpeak 3 supports Opus for voice and music from server 3.0.7 and client version 3.0.10 [40][41]
  • Bria X-Lite supports Opus
  • Acrobits Softphone supports Opus

Web frameworks and browsers[edit]

  • Opus support is mandatory for WebRTC implementations.[42]
  • Mozilla supports Opus beginning with version 15 of Firefox and Thunderbird.[43]
  • Depending on the backend in use, Opera supports inline playback of embedded Opus files.[44] Official support for Opus and WebRTC are on the development roadmap.[45]
  • Chromium and Google Chrome supports Opus audio as of version 26.[46] It is however disabled by default for embedded streams.

Streaming audio[edit]

  • Icecast.[47] There are also streaming Icecast radio trials that have been live since September 2012[48] and January 2013.[49]
  • Liquidsoap
  • Airtime includes Opus support by default since version 2.4.0.[50]
  • Xeocoder is a dedicated Opus encoder for live streaming with support for professional automation systems.

Operating systems and desktop multimedia frameworks[edit]

  • In Debian GNU/Linux, Arch Linux and Ubuntu the Opus development tools and supporting libraries can be installed from the preconfigured repositories.[51]
  • For Microsoft Windows, there are DirectShow filters supporting Opus, including DC-Bass Source Mod and the LAV Filters.[52]
  • In GStreamer the integration of Opus support is complete.[53]
  • FFmpeg supports decoding and encoding Opus via the external library libopus since FFmpeg 1.1.[54]
  • SteamOS uses Opus or Vorbis for streaming audio

Hardware devices and embedded support[edit]

  • The GatesAir Intraplex IP Link supports Opus as of version 2.1.
  • The Tieline Merlin and Genie families of broadcast IP audio codecs support Opus, as do the Report-IT Enterprise Android and iOS soft-codec applications.

Player software (Desktop)[edit]

  • AIMP supports Opus natively as of version 3.20.[57]
  • Amarok supports Opus as of version 2.8.[58]
  • CMUS supports Opus from version 2.5.0
  • DeaDBeeF (Linux) supports Opus (via ffmpeg) as of version 0.6.0.[59]
  • foobar2000 supports the format natively as of v1.1.14 beta 1.[60]
  • Mpxplay supports Opus (using a decoder DLL) as of v1.60 alpha 2
  • MusicBee supports Opus natively as of version 2.0.[61]
  • Swinsian (a music player for Mac OS X) supports Opus natively as of version 1.8.9.[62]
  • VLC media player supports Opus since version 2.0.4[63]
  • Vox Music Player - Mac OS X app has playback support, but doesn't display metadata.
  • Winamp supports Opus (using a third-party DLL)[64]
  • Xmplay supports Opus (using a decoder DLL) as of v3.6; with libopus 1.1 support in v3.8

Player software (Smartphone)[edit]

Other software[edit]

  • Asunder - an audio CD ripper
  • CDBurnerXP
  • Libav
  • MediaCoder
  • SFBAudioEngine - audio playback engine for Mac OS X and iOS
  • TAudioConverter - audio converter for Windows supports converting from, and to Opus
  • X Lossless Decoder - an audio CD ripper and format converter for Mac OS X

See also[edit]

References[edit]

  1. ^ a b "Ogg Encapsulation for the Opus Audio Codec". IETF. 2012-07-16. Retrieved 2012-09-12. 
  2. ^ Network Working Group (July 4, 2011). "RTP Payload Format and File Storage Format for Opus Speech and Audio Codec". Opus codec. IETF. Retrieved October 26, 2011. 
  3. ^ a b c "Opus Codec". Opus (Home page). Xiph.org Foundation. Retrieved July 31, 2012. 
  4. ^ http://arstechnica.com/gadgets/2012/09/newly-standardized-opus-audio-codec-fills-every-role-from-online-chat-to-music/
  5. ^ a b Raymond Chen et al. Opus Testing. IETF 80
  6. ^ Terriberry, Timothy (July 19, 2012). "Firefox Beta 15 supports the new Opus audio format". Mozilla Hacks. Mozilla Foundation. Retrieved July 31, 2012. 
  7. ^ a b "It's Opus, it rocks and now it's an audio codec standard!". Mozilla Hacks. Retrieved 2012-09-12. 
  8. ^ "MKVToolNix v6.4.0 released". Matroska users mailing list. 2013-09-16. http://lists.matroska.org/pipermail/matroska-users/2013-September/006883.html. Retrieved 2013-12-24.
  9. ^ "IETF working towards royalty-free audio codec". H-online.com. 2009-11-13. Archived from the original on 25 January 2013. Retrieved 2012-09-12. 
  10. ^ Vos, Koen (2010-10-15). "Harmony became Opus". ietfcodec Discussion Archive mailing list. http://www.ietf.org/mail-archive/web/codec/current/msg01852.html. Retrieved 2011-06-19.
  11. ^ "[codec] Harmony became Opus". Ietf.org. 2010-10-15. Retrieved 2012-09-12. 
  12. ^ "IETF Opus codec now ready for testing". Hydrogenaudio.org. Retrieved 2012-09-12. 
  13. ^ Valin, Jean-Marc (2011-08-01). "IETF update, Mozilla". LiveJournal.com. Retrieved 2012-10-05. 
  14. ^ "Opus Codec". Opus Codec. Retrieved 2012-09-12. 
  15. ^ "Opus approved by the IETF". Jmspeex.livejournal.com. 2012-07-03. Retrieved 2012-09-12. 
  16. ^ "[opus] Release candidates for 1.0.0 and 1.0.1 are available". Lists.xiph.org. Retrieved 2012-10-05. 
  17. ^ "RFC 6716 on Definition of the Opus Audio Codec". Ietf.org. Retrieved 2012-09-12. 
  18. ^ Maxwell, Gregory (2011). "64kbit/sec stereo multiformat listening test — unofficial results page". Xiph.Org Foundation. Retrieved 2011-06-19. 
  19. ^ Next-Gen Low-Latency Open Codec Beats HE-AAC, Slashdot-Meldung vom 14. April 2011
  20. ^ Valin, Jean-Marc; Vos, Koen; Skoglund, Jan (2011-10-24). Hoene, Christian, ed. "Summary of Opus listening test results". Internet Engineering Task Force. Retrieved 2012-01-16. 
  21. ^ Lutzky, Manfred; Schuller, Gerald; Gayer, Marc; Krämer, Ulrich; Wabnik, Stefan (2004). "A guideline to audio codec delay". In AES 116th convention, Berlin, Germany: 8–11. Retrieved 30 October 2012. 
  22. ^ Lironi, F.; et al. (2005). "Multi RAB-based multimedia services over GERAN mobile networks". Vehicular Technology Conference, VTC-2005-Fall. IEEE 62nd 3: 1662–1666. Retrieved 30 October 2012. 
  23. ^ Lago, Nelson Posse; Kon, Fabio (November 2004). "The quest for low latency". In Proceedings of the International Computer Music Conference: 33–36. Retrieved 30 October 2012. 
  24. ^ Carôt, Alexander (2010). "Low Latency Audio Streaming for Internet-Based Musical Interaction." in Streaming Media Architectures: Techniques and Applications: Recent Advances. IGI Global. pp. 362–383. ISBN 978-1-61692-833-9. 
  25. ^ YE, NONG, VAN CHEN, and TONI FARLEY (2003). "Qos Requirements Of Multimedia Data On Computer Networks." Proceedings of the Second International Conference on Active Media Technology, Chongqing, PR China, 29-31 May 2003. World Scientific Publishing Company Incorporated. pp. 183–189. ISBN 978-981-238-343-3. 
  26. ^ Montgomery, Christopher. "A quick showcase of the bleeding edge... CELT 0.10.0 @ constant PEAQ value, varying latency". CELT v0.10 (latest prior to Opus integration). xiph.org. Retrieved 30 October 2012. 
  27. ^ Valin, Jean-Marc et al. "Opus Codec Overview". IETF RFC 6716. IETF. Retrieved 30 October 2012. 
  28. ^ http://lists.digium.com/pipermail/asterisk-dev/2013-May/060419.html
  29. ^ https://github.com/meetecho/asterisk-opus
  30. ^ "Neues von Mumble: Snapshots mit Opus-Codec verfügbar". Natenom.name. 2010-10-14. Retrieved 2012-09-12. 
  31. ^ "1.2.4 - Mumble". Mumble. Retrieved 2012-09-12. 
  32. ^ Mumble-Server (Murmur) Opus-Codec
  33. ^ Mumble (Client) Next generation audio codec
  34. ^ "SFLphone — Task #14602: [Codec] Implement opus". Savoir-faire Linux. 2012-08-13. Retrieved 2012-09-12. 
  35. ^ TrueConf supports Opus
  36. ^ "(SIP Communicator) | GSOC / Opus and CELT support in". Jitsi. Retrieved 2012-09-12. 
  37. ^ "(SIP Communicator) | Development / Roadmap". Jitsi. Retrieved 2012-09-12. 
  38. ^ https://download.jitsi.org/jitsi/changelogs/changelog-4506.html
  39. ^ "A new Line2 for iOS experience with improved sound quality". Line2. 2012-10-18. Retrieved 2013-03-15. 
  40. ^ "TeamSpeak 3 Client 3.0.10 released". TeamSpeak Forums. Retrieved 2013-03-04. 
  41. ^ "TeamSpeak 3 Server Version 3.0.7 released". TeamSpeak Forums. Retrieved 2013-03-04. 
  42. ^ "Zwei Audio-Codecs für Echtzeit-Kommunikation im Browser". Heise.de. Retrieved 2012-09-12. 
  43. ^ "Media formats supported by the HTML audio and video elements". Developer.mozilla.org. 2012-09-03. Retrieved 2012-09-12. 
  44. ^ "Why no official mention of Opus support?". My.opera.com. 2012-07-19. Retrieved 2012-09-12. 
  45. ^ Shankland, Stephen (2012-08-17). "How corporate bickering hobbled better Web audio". CNET News. Retrieved 2012-09-12. 
  46. ^ https://code.google.com/p/chromium/issues/detail?id=104241
  47. ^ "Icecast 2.4 beta release". Lists.xiph.org. Retrieved 2012-09-12. 
  48. ^ Absolute Radio, Listen Labs. "Opus Streaming Trial". Absolute Radio. Retrieved 30 October 2012. 
  49. ^ Radio Jackie, Listen Now. "Opus Stream". Radio Jackie. Retrieved 14 May 2013. 
  50. ^ Airtime, Changelog. "Airtime/changelog at 2.4.x". Sourcefabric. Retrieved 1 July 2013. 
  51. ^ Debian Webmaster, webmaster@debian.org. "Details of package opus-tools". packages.debian.org. Retrieved 2013-05-07. 
  52. ^ "lavfilters — Open-Source DirectShow Media Splitter and Decoders". Google Project Hosting. Retrieved 2012-10-05. 
  53. ^ "GStreamer: news". Gstreamer.freedesktop.org. Retrieved 2012-10-05. 
  54. ^ http://ffmpeg.org/download.html#release_1.1
  55. ^ Rockbox Contributors (2013-03-05). "Release notes for Rockbox v.3.13". Rockbox. Retrieved 2013-03-21. 
  56. ^ "RockboxAsAnApplication2010 < Main < Wiki". Rockbox.org. Retrieved 2012-10-05. 
  57. ^ "AIMP: Home Page". Aimp.ru. Retrieved 2012-10-05. 
  58. ^ http://amarok.kde.org/en/releases/2.8
  59. ^ http://deadbeef.sourceforge.net/posts/deadbeef_0.6.0_is_out.html
  60. ^ "foobar2000 v1.1.14 beta". Hydrogenaudio Forums. Retrieved 2012-10-05. 
  61. ^ "MusicBee". MusicBee Forum. Retrieved 2013-08-17. 
  62. ^ "Swinsian". Swinsian - Changelog. 2013-10-30. Retrieved 2013-10-31. 
  63. ^ "VLC 2.0.4 Twoflower". VideoLAN. Retrieved 2012-10-19. 
  64. ^ "Winamp". Winamp Forums. Retrieved 2013-01-04. 
  65. ^ "Capriccio". Junho Choi, Seungjoon Lee. Retrieved 2013-08-13. 

External links[edit]