Opus (audio format)
|Internet media type||
|Developed by||IETF codec working group|
|Initial release||September 11, 2012|
|Type of format||Audio|
|Contained by||Ogg, Matroska, WebM, MPEG-TS|
|Extended from||SILK, CELT|
|Initial release||August 26, 2012|
1.1.3/ July 15, 2016
|Type||Audio codec, reference implementation|
|License||3-clause BSD license|
|Website||Opus codec downloads|
Opus is a lossy audio coding format developed by the Xiph.Org Foundation and standardized by the Internet Engineering Task Force, designed to efficiently code speech and general audio in a single format, while remaining low-latency enough for real-time interactive communication and low-complexity enough for low end ARM3 processors. Opus replaces both Vorbis and Speex for new applications, and several blind listening tests have ranked it higher quality than any other standard audio format at any given bitrate until transparency is reached, including MP3, AAC, and HE-AAC.
Opus combines the speech-oriented linear predictive coding SILK algorithm, and the lower-latency, MDCT-based CELT algorithm, switching between or combining them as needed for maximum efficiency. Bitrate, audio bandwidth, complexity, and algorithm can all be adjusted seamlessly in each frame. Opus has the low algorithmic delay (26.5 ms by default) necessary for use as part of a real-time communication link, permitting natural conversation, networked music performances, and live lip sync; by trading-off quality or bitrate the delay can be reduced down to 5 ms. Its delay is exceptionally low compared to competing codecs, which require well over 100 ms, yet Opus performs very competitively with these formats in terms of quality per bitrate.
As an open format standardized through RFC 6716, a reference implementation called libopus is available under the New BSD License. The reference has both fixed-point and floating-point optimizations for low- and high-end devices, with SIMD optimizations on platforms that support them. All known software patents which cover Opus are licensed under royalty-free terms.
Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and five sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, the human hearing range). An Opus stream can support up to 255 audio channels, and it allows channel coupling between channels in groups of two using mid-side coding.
Opus has very short latency (26.5 ms by default), which makes it suitable for real-time applications such as telephony, Voice over IP and videoconferencing; research by Xiph led to the CELT codec, which allows the highest quality while maintaining low delay. In any Opus stream, the bitrate, bandwidth, and delay can be continually varied without introducing any distortion or discontinuity; even mixing packets from different streams will cause a smooth change, rather than the distortion common in other codecs. Unlike Vorbis, Opus does not require large codebooks for each individual file, making it more efficient for short clips of audio and more resilient.
As an open standard, the algorithms are openly documented, and a reference implementation (including the source code) is published. Broadcom and the Xiph.Org Foundation own software patents on some of the CELT algorithms, and Skype Technologies/Microsoft own some on the SILK algorithms; each offers a royalty-free perpetual for use with Opus, reserving only the right to make use of their patents to defend against infringement suits of third parties. Qualcomm, Huawei, France Telecom, and Ericsson have claimed that their patents may apply, which Xiph's legal counsel denies, and none have pursued any legal action. The Opus license automatically and retroactively terminates for any entity that attempts to file a patent suit.
The Opus format is based on a combination of the full-bandwidth CELT format and the speech-oriented SILK format, both heavily modified: CELT is based on the MDCT that most music codecs use, using CELP techniques in the frequency domain for better prediction, while SILK uses linear predictive coding (LPC) and an optional Long-Term Prediction filter to model speech. In Opus, both were modified to support more frame sizes, as well as further algorithmic improvements and integration, such as using CELT's range encoder for both types. To minimize packet overhead at low bitrates, if latency is not as pressing, SILK has support for packing multiple 20 ms frames together, sharing context and headers; SILK also allows Low Bit-Rate Redundancy (LBRR) frames, allowing low-quality packet loss recovery. CELT includes both spectral replication and noise generation, similar to AAC's SBR and PNS, and can further save bits by filtering out all harmonics of tonal sounds entirely, then replicating them in the decoder. Better tone detection is an ongoing project to improve quality.
The format has three different modes: speech, hybrid, and CELT. The basic speech mode is pure SILK, up to 8 kHz, while the hybrid speech mode combines SILK for the speech and uses CELT for the frequency range above 8 kHz, allowing an easy fallback to pure SILK at very low bitrates. The third mode is pure-CELT, designed for general audio. SILK is inherently VBR and cannot hit a bitrate target, while CELT can always be encoded to any specific number of bytes, enabling hybrid and CELT mode when CBR is required.
The reference implementation is written in C and compiles on hardware architectures with or without a floating-point unit, although floating-point is currently required for audio bandwidth detection (dynamic switching between SILK, CELT, and hybrid encoding) and most speed optimizations.
Opus was originally specified for encapsulation in Ogg containers, specified as
audio/ogg; codecs=opus, and for Ogg Opus files the
.opus filename extension is recommended. Matroska, WebM, MPEG-TS, and MP4 all officially support Opus streams.
|Abbreviation||Audio bandwidth||Effective sample rate|
|NB (narrowband)||4 kHz||8 kHz|
|MB (medium-band)||6 kHz||12 kHz|
|WB (wideband)||8 kHz||16 kHz|
|SWB (super-wideband)||12 kHz||24 kHz|
|FB (fullband)||20 kHz[nb 1]||48 kHz|
Opus was proposed for the standardization of a new audio format at the IETF, which was eventually accepted and granted by the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A. (now Microsoft). Its main developers are Jean-Marc Valin (Xiph.Org, Octasic, Mozilla Corporation), Koen Vos (Skype), and Timothy B. Terriberry (Xiph.Org, Mozilla Corporation). Among others, Juin-Hwey (Raymond) Chen (Broadcom), Gregory Maxwell (Xiph.Org, Wikimedia), and Christopher Montgomery (Xiph.Org) were also involved.
The development of the CELT part of the format goes back to thoughts on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.
The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.
In March 2009, Skype suggested the development and standardization of a wideband audio format within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group. Representatives of several companies which were taking part in the standardization of patent-encumbered competing formats stated objections against the start of the standardization process for a royalty-free format: representatives of Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs (department of France Télécom), which were involved in the creation of G.718. The working group finally formed in February 2010, and even the corresponding Study Group 16 from the ITU-T pledged to support its work.
In July 2010, a prototype of a hybrid format was presented that combined the two proposed format candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010. At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes. Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus. In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012. On July 2, 2012, Opus was approved by the IETF for standardization. The reference software entered release candidate state on August 8. The final specification was released as RFC 6716 on September 10, 2012. and versions 1.0 and 1.0.1 of the reference implementation were released the day after.
On December 5, 2013, Opus 1.1 was released, incorporating overall speed improvements and significant encoder quality improvements: Tonality estimation boosts bitrate and quality for previously problematic samples, like harpsichords; automated speech/music detection improves quality in mixed audio; mid-side stereo reduces the bitrate needs of many songs; band precision boosting for improved transients; and DC rejection below 3 Hz. Two new VBR modes were added: unconstrained for more consistent quality, and temporal VBR that boosts louder frames and generally improves quality.
Opus 1.1.1 was released on November 26, 2015, and 1.1.2 on January 12, 2016, both adding speed optimizations and bug fixes. July 15, 2016 saw the release of version 1.1.3 and includes bugfixes, optimalizations, documentation updates and experimental Ambisonics work.
Quality comparison and low latency performance
In listening tests around 64 kbit/s, Opus shows superior quality compared to HE-AAC codecs, which were previously dominant due to their use of the patented spectral band replication (SBR) technology. In listening tests around 96 kbit/s, Opus shows slightly superior quality compared to AAC, and significantly better quality compared to Vorbis and MP3.
Opus has very low algorithmic delay, a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Total algorithmic delay for an audio format is the sum of delays that must be incurred in the encoder and the decoder of a live audio stream regardless of processing speed and transmission speed, such as buffering audio samples into blocks or frames, allowing for window overlap and possibly allowing for noise-shaping look-ahead in a decoder and any other forms of look-ahead, or for an MP3 encoder, the use of bit reservoir.
Total one-way latency below 150 ms is the preferred target of most VoIP systems, to enable natural conversation with turn-taking little affected by delay. Musicians typically feel in-time with up to around 30 ms audio latency, roughly in accord with the fusion time of the Haas effect, though matching playback delay of each user's own instrument to the round-trip latency can also help. It is suggested for lip sync that around 45-100 ms audio latency may be acceptable.
Opus permits trading-off reduced quality or increased bitrate to achieve an even smaller algorithmic delay (5.0 ms minimum). While the reference implementation's default Opus frame is 20.0 ms long, the SILK layer requires a further 5.0 ms lookahead plus 1.5 ms for resampling, giving a default delay of 26.5 ms. When the CELT layer is active, it requires 2.5 ms lookahead for window overlap to which a matching delay of 4.0 ms is added by default to synchronize with the SILK layer. If the encoder is instantiated in the special restricted low delay mode, the 4.0 ms matching delay is removed and the SILK layer is disabled, permitting the minimum algorithmic delay of 5.0 ms.
The first is uncompressed wav file. The second is a mp3 file encoded at 16kbps using LAME, and third is an opus file encoded at 16kbps.
|Problems playing this file? See media help.|
The format and algorithms are openly documented and the reference implementation is published as free software. Xiph's reference implementation is called libopus, along with executables called opus-tools, and it consists of a separate encoder and decoder; it is published under the terms of a BSD-like license. It is written in C and can be compiled for hardware architectures with or without a floating point unit. The accompanying diagnostic tool opusinfo reports detailed technical information about Opus files, including information on the standard compliance of the bitstream format. It is based on ogginfo from the vorbis-tools and therefore — unlike the encoder and decoder — is available under the terms of version 2 of the GPL.
Concentus is an implementation in C#, with plans to implement Java in the future. It is technically a port of the libopus reference library; at present, it directly ports the libopus C code to C#, including emulating pointers, and so is currently significantly slower speed than the C library.
Most end-user software relies on multimedia frameworks provided by the operating system. Native Opus codec support is implemented in most major multimedia frameworks for Unix-like operating systems, including GStreamer, FFmpeg, and Libav libraries. Although Windows DirectShow lacks an Opus decoder, the third-party LAV Filters adds support. Google's Android supports the Opus audio codec natively since version 5.0 ("Lollipop"), though built-in players require it to be encapsulated in Matroska or Ogg containers.
While support in multimedia frameworks automatically enables Opus support in software which is built on top of such frameworks, several applications developers made additional efforts for supporting the Opus audio format in their software. Such support was added to AIMP, Amarok, cmus, foobar2000, Mpxplay, MusicBee, SMplayer, VLC media player, Winamp and Xmplay audio players; Icecast, Airtime audio streaming software; and Asunder audio CD ripper, CDBurnerXP CD burner, FFmpeg, Libav and MediaCoder media encoding tools. Streaming Icecast radio trials are live since September 2012 and January 2013. SteamOS uses Opus or Vorbis for streaming audio.
Opus support is mandatory for WebRTC implementations. Opus is supported in Mozilla Firefox, Chromium and Google Chrome, Blink-based Opera, as well as all browsers for Unix-like systems relying on GStreamer for multimedia formats support. Although Internet Explorer will not provide Opus playback natively, support for the format is being built into the Edge browser, along with VP9, for full WebM support.
Due to its abilities, Opus gained early interest from VoIP software vendors. Several SIP clients, including Acrobits Softphone, Bria X-Lite, CSipSimple (via additional plug-in), Empathy (via GStreamer), Jitsi, Tuenti, Line2 (currently only on iOS), Linphone, Phoner and PhonerLite, SFLphone, Mumble and TeamSpeak 3 voice chat software also support Opus. TrueConf supports Opus in its VoIP products. Asterisk lacks builtin Opus support for legal reasons, but third-party patch is available for download. Tox P2P videoconferencing software uses Opus exclusively. Classified-ads distributed messaging app sends raw opus frames inside TLS socket in its VoIP implementation.
Since version 3.13 Rockbox enables Opus playback on supported portable media players, including some products from the iPod series by Apple, devices made by iriver, Archos and Sandisk, and on Android devices using "Rockbox as an Application." Grandstream GXV3240 and GXV3275 video IP phones support Opus audio both for encoding and decoding. OBihai OBi1062, OBi1032 and OBi1022 IP phones all support Opus.
- Opus cuts audio above 20 kHz, the generally accepted upper limit of the human hearing range.
- Spittka, Julian; Vos, Koen; Valin, Jean-Marc (2015-06-30). "Opus Media Type Registration". RTP Payload Format for the Opus Speech and Audio Codec. IETF. p. 9. sec. 6.1. ISSN 2070-1721. RFC 7587. https://tools.ietf.org/html/rfc7587#section-6.1. Retrieved 2015-06-30.
- Terriberry, Timothy; Lee, Ron; Giles, Ralph (April 2016). "Content Type". Ogg Encapsulation for the Opus Audio Codec. IETF. p. 30. sec. 9. RFC 7845. https://tools.ietf.org/html/rfc7845#section-9. Retrieved 2016-04-30.
- "Opus Codec". Opus (Home page). Xiph.org Foundation. Retrieved July 31, 2012.
- Bright, Peter (2012-09-12). "Newly standardized Opus audio codec fills every role from online chat to music". Ars Technica. Retrieved 2014-05-28.
- Valin, Jean-Marc; Vos, Koen; Skoglund, Jan (2013-05-17). Hoene, Christian, ed. "Summary of Opus listening test results". Internet Engineering Task Force. Retrieved 2015-06-30.
- Kamedo2 (July 2014). "Results of the public multiformat listening test".
- Jean-Marc Valin; Gregory Maxwell; Timothy B. Terriberry; Koen Vos (October 17–20, 2013). "High-Quality, Low-Delay Music Coding in the Opus Codec" (PDF). www.xiph.org. New York, USA: Xiph.Org Foundation. p. 2. Retrieved 19 August 2014.
CELT's look-ahead is 2.5 ms, while SILK's look-ahead is 5 ms, plus 1.5 ms for the resampling (including both encoder and decoder resampling). For this reason, the CELT path in the encoder adds a 4 ms delay. However, an application can restrict the encoder to CELT and omit that delay. This reduces the total look-ahead to 2.5 ms.
- Chen, Raymond (April 1, 2011). "Opus Testing" (PDF). Retrieved 2016-01-02.
- "Opus Codec License". Xiph.org. Retrieved 2015-12-22.
- "Nokia's VP8 patent claims: we've been here before". OSNews. 2013-05-25. Retrieved 2014-07-17.
- Montgomery, Christopher (December 23, 2010). "next generation audio: CELT update 20101223".
- Bunkus, Moritz (2013-09-16). "MKVToolNix v6.4.0 released". Matroska users (Mailing list). Retrieved 2013-12-24.
- "WebM Container Guidelines". The WebM Project. Retrieved 19 October 2015.
- "Registration for format_identifier Opus". SMPTE. 2014-05-05. Retrieved 2016-01-03.
- "Sample Entry Codes Registered to ISO". MP4REG. Retrieved 2016-01-03.
- Valin, Jean-Marc; Vos, Koen; Terriberry, Timothy B. (2012-09-11). "Opus Codec Overview". Definition of the Opus Audio Codec. p. 7. sec. 2.. ISSN 2070-1721. RFC 6716. https://tools.ietf.org/html/rfc6716#section-2.. Retrieved 2014-06-21.
- "IETF working towards royalty-free audio codec". H-online.com. 2009-11-13. Retrieved 2016-04-07.
- Vos, Koen (2010-10-15). "Harmony became Opus". codec (Mailing list). IETF. Retrieved 2016-01-03.
- "IETF Opus codec now ready for testing". Hydrogenaudio. Retrieved 2012-09-12.
- Valin, Jean-Marc (2011-08-01). "IETF update, Mozilla". LiveJournal.com. Retrieved 2012-10-05.
- "Opus Codec". Xiph.org Foundation. Retrieved 2012-09-12.
- "Opus approved by the IETF". Jmspeex.livejournal.com. 2012-07-03. Retrieved 2012-09-12.
- "[opus] Release candidates for 1.0.0 and 1.0.1 are available". opus (Mailing list). Xiph.org Foundation. Retrieved 2012-10-05.
- "It's Opus, it rocks and now it's an audio codec standard!". Mozilla Hacks. Retrieved 2012-09-12.
- "RFC 6716 on Definition of the Opus Audio Codec". announce (Mailing list). IETF. Retrieved 2012-09-12.
- Monty; Valin, Jean-Marc. "Opus update 20131205: 1.1 Release".
- Maxwell, Gregory (2011). "64kbit/sec stereo multiformat listening test — unofficial results page". Xiph.Org Foundation. Retrieved 2011-06-19.
- Next-Gen Low-Latency Open Codec Beats HE-AAC, Slashdot-Meldung vom 14. April 2011
- Lutzky, Manfred; Schuller, Gerald; Gayer, Marc; Krämer, Ulrich; Wabnik, Stefan (2004). "A guideline to audio codec delay" (PDF). In AES 116th convention, Berlin, Germany: 8–11. Retrieved 30 October 2012.
- Lironi, F.; Masseroni, C.; Trivisonno, R.; Ball, C.F. (2005). "Multi RAB-based multimedia services over GERAN mobile networks". Vehicular Technology Conference, VTC-2005-Fall. IEEE 62nd. 3: 1662–1666. Retrieved 30 October 2012.
- Lago, Nelson Posse; Kon, Fabio (November 2004). "The quest for low latency" (PDF). In Proceedings of the International Computer Music Conference: 33–36. Retrieved 30 October 2012.
- Carôt, Alexander (2010). "Low Latency Audio Streaming for Internet-Based Musical Interaction." in Streaming Media Architectures: Techniques and Applications: Recent Advances. IGI Global. pp. 362–383. ISBN 978-1-61692-833-9.
- YE, NONG, VAN CHEN, and TONI FARLEY (2003). "Qos Requirements Of Multimedia Data On Computer Networks." Proceedings of the Second International Conference on Active Media Technology, Chongqing, PR China, 29-31 May 2003. World Scientific Publishing Company Incorporated. pp. 183–189. ISBN 978-981-238-343-3.
- Montgomery, Christopher. "A quick showcase of the bleeding edge... CELT 0.10.0 @ constant PEAQ value, varying latency". CELT v0.10 (latest prior to Opus integration). xiph.org. Retrieved 30 October 2012.
- "GStreamer: news". Gstreamer.freedesktop.org. Retrieved 2012-10-05.
- "Audio Codecs". FFmpeg General Documentation. Retrieved 2014-05-28.
- "Audio Codecs". Libav General Documentation. Retrieved 2014-05-28.
- "lavfilters — Open-Source DirectShow Media Splitter and Decoders". Google Project Hosting. Retrieved 2012-10-05.
- "Supported Media Formats". Google. Retrieved 2015-04-29.
- Google. "Android Lollliop, Android Developers". android.com. Retrieved 2015-11-03.
- Google. "Issue 80729: Opus support not working.". code.google.com. Retrieved 2016-04-19.
- "AIMP: Home Page". Aimp.ru. Retrieved 2012-10-05.
- "Amarok 2.8 "Return To The Origin" released". Amarok. 2013-08-16. Retrieved 2014-05-28.
- "foobar2000 v1.1.14 beta". Hydrogenaudio Forums. Retrieved 2012-10-05.
- "MusicBee". MusicBee Forum. Retrieved 2013-08-17.
- "VLC 2.0.4 Twoflower". VideoLAN. Retrieved 2012-10-19.
- "Winamp". Winamp Forums. Retrieved 2013-01-04.
- "Icecast 2.4 beta release". Lists.xiph.org. Retrieved 2012-09-12.
- Airtime, Changelog. "Airtime/changelog at 2.4.x". Sourcefabric. Retrieved 1 July 2013.
- Radio Jackie, Listen Now. "Opus Stream". Radio Jackie. Retrieved 14 May 2013.
- Absolute Radio, Listen Labs. "Opus Streaming Trial". Absolute Radio. Retrieved 30 October 2012.
- "Steam Beta Update: SteamVR and In-Home Streaming Beta". SteamDB. Retrieved 9 December 2015.
- "Zwei Audio-Codecs für Echtzeit-Kommunikation im Browser". Heise.de. Retrieved 2012-09-12.
- "Media formats supported by the HTML audio and video elements". Developer.mozilla.org. 2012-09-03. Retrieved 2012-09-12.
- "Issue 104241: Support OPUS in Ogg files for the audio tag and Audio object". Chromium bug tracker. Retrieved 2014-05-28.
- "Why no official mention of Opus support?". My.opera.com. 2012-07-19. Retrieved 2012-09-12.
- Shankland, Stephen (2012-08-17). "How corporate bickering hobbled better Web audio". CNET News. Retrieved 2012-09-12.
- "Platform status". Microsoft. 2016-04-19.
- "WebM, VP9 and Opus Support in Microsoft Edge". Microsoft. 2016-04-18.
- Bocquet, Ludovic (2013-03-07). "Jitsi 2.0 Now Released!". XMPP Foundation. Retrieved 2014-05-28.
- "Tuenti+WebRTC (Voip2day 2014)".
- "A new Line2 for iOS experience with improved sound quality". Line2. 2012-10-18. Retrieved 2013-03-15.
- "Linphone technical features".
- "Phoner Configuration".
- "SFLphone — Task #14602: [Codec] Implement opus". Savoir-faire Linux. 2012-08-13. Retrieved 2012-09-12.
- Brandon (2013-06-19). "Mumble 1.2.4 Has been released". Mumblevoice Blog. Retrieved 2014-05-28.
- "TeamSpeak 3 Client 3.0.10 released". TeamSpeak Forums. Retrieved 2013-03-04.
- "TeamSpeak 3 Server Version 3.0.7 released". TeamSpeak Forums. Retrieved 2013-03-04.
- "TrueConf сделал ставку на аудиокодек Opus" [TrueConf opted for Opus audio codec] (Press release) (in Russian). Moscow, Russia. 2012-09-19. Retrieved 2014-05-28.
- Jordan, Matthew (2013-05-29). "Opus and VP8". asterisk-dev (Mailing list). Retrieved 2014-05-28.
- "Asterisk Opus/VP8 patch". GitHub. Retrieved 2014-05-28.
- "Tox codec handling source code". GitHub. Retrieved 2015-12-09.
- "Classified-ads audio encoder documentation". Retrieved 2016-05-25.
- "Tieline Integrates OPUS into Report-IT and IP codecs". Radio Magazine. 2012-12-18. Retrieved 2012-12-18.
- Rockbox Contributors (2013-03-05). "Release notes for Rockbox v.3.13". Rockbox. Retrieved 2013-03-21.
- "RockboxAsAnApplication2010 < Main < Wiki". Rockbox.org. Retrieved 2012-10-05.
|Wikimedia Commons has media related to Opus.|