Jump to content

Video coding format: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Line 17: Line 17:


==History==
==History==
===Video compression===
The concept of [[analog video]] compression dates back to 1929, when R.D. Kell in Britain proposed the concept of transmitting only the portions of the scene that changed from frame-to-frame. The concept of [[digital video]] compression dates back to 1952, when [[Bell Labs]] researchers B.M. Oliver and C.W. Harrison proposed the use of [[differential pulse-code modulation]] (DPCM) in video coding. In 1959, [[NHK]] researchers Y. Taki, M. Hatori and S. Tanaka proposed [[inter-frame]] video coding.<ref>{{cite web |title=History of Video Compression |url=https://www.itu.int/wftp3/av-arch/jvt-site/2002_07_Klagenfurt/JVT-D068.doc |website=[[ITU-T]] |publisher=Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) |date=July 2002 |pages=11, 27-9, 53 |accessdate=3 November 2019}}</ref> In 1967, [[University of London]] researchers A.H. Robinson and C. Cherry proposed [[run-length encoding]] (RLE), a [[lossless compression]] scheme, to slightly reduce the transmission bandwidth of [[analog television]] signals.<ref name="robinson">{{cite journal |author1-last=Robinson |author1-first=A. H. |author2-last=Cherry |author2-first=C. |title=Results of a prototype television bandwidth compression scheme |journal=[[Proceedings of the IEEE]] |publisher=[[IEEE]] |volume=55 |number=3 |date=1967 |pages=356–364 |doi=10.1109/PROC.1967.5493}}</ref>
The concept of [[analog video]] compression dates back to 1929, when R.D. Kell in Britain proposed the concept of transmitting only the portions of the scene that changed from frame-to-frame. The concept of [[digital video]] compression dates back to 1952, when [[Bell Labs]] researchers B.M. Oliver and C.W. Harrison proposed the use of [[differential pulse-code modulation]] (DPCM) in video coding. The concept of [[motion compensation]] dates back to 1959, when [[NHK]] researchers Y. Taki, M. Hatori and S. Tanaka proposed [[inter-frame]] video coding.<ref name="ITU">{{cite web |title=History of Video Compression |url=https://www.itu.int/wftp3/av-arch/jvt-site/2002_07_Klagenfurt/JVT-D068.doc |website=[[ITU-T]] |publisher=Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) |date=July 2002 |pages=11, 24-9, 33, 53-5 |accessdate=3 November 2019}}</ref> In 1967, [[University of London]] researchers A.H. Robinson and C. Cherry proposed [[run-length encoding]] (RLE), a [[lossless compression]] scheme, to slightly reduce the transmission bandwidth of [[analog television]] signals.<ref name="robinson">{{cite journal |author1-last=Robinson |author1-first=A. H. |author2-last=Cherry |author2-first=C. |title=Results of a prototype television bandwidth compression scheme |journal=[[Proceedings of the IEEE]] |publisher=[[IEEE]] |volume=55 |number=3 |date=1967 |pages=356–364 |doi=10.1109/PROC.1967.5493}}</ref>


The earliest [[digital video]] coding algorithms were either for [[uncompressed video]] or used [[lossless compression]], both methods inefficient and impractical for digital video coding.<ref name="Ghanbari">{{cite book |last1=Ghanbari |first1=Mohammed |title=Standard Codecs: Image Compression to Advanced Video Coding |date=2003 |publisher=[[Institution of Engineering and Technology]] |isbn=9780852967102 |pages=1-2 |url=https://books.google.com/books?id=7XuU8T3ooOAC&pg=PA1}}</ref><ref name="Lea">{{cite book |last1=Lea |first1=William |title=Video on demand: Research Paper 94/68 |date=1994 |publisher=[[House of Commons Library]] |location=9 May 1994 |url=https://researchbriefings.parliament.uk/ResearchBriefing/Summary/RP94-68 |accessdate=20 September 2019}}</ref> Experimental digital video systems began development in the 1970s,<ref name="Ghanbari"/> initially using uncompressed [[pulse-code modulation]] (PCM) requiring high [[bitrate]]s around 45{{ndash}}200 [[Mbps]] for [[standard-definition]] (SD) video,<ref name="Ghanbari"/><ref name="Lea"/> which was up to 2,000 times greater than the [[telecommunication]] [[Bandwidth (computing)|bandwidth]] (up to 100{{nbsp}}[[kilobits per second|kbps]]) available until the 1990s.<ref name="Lea"/> Similarly, uncompressed [[high-definition video|high-definition]] (HD) [[1080p]] video requires bitrates exceeding 1{{nbsp}}[[Gbps]], significantly greater than the bandwidth available in the 2000s.<ref>{{cite book |last1=Lee |first1=Jack |title=Scalable Continuous Media Streaming Systems: Architecture, Design, Analysis and Implementation |date=2005 |publisher=[[John Wiley & Sons]] |isbn=9780470857649 |page=25 |url=https://books.google.com/books?id=7fuvu52cyNEC&pg=PA25}}</ref>
The earliest [[digital video]] coding algorithms were either for [[uncompressed video]] or used [[lossless compression]], both methods inefficient and impractical for digital video coding.<ref name="Ghanbari">{{cite book |last1=Ghanbari |first1=Mohammed |title=Standard Codecs: Image Compression to Advanced Video Coding |date=2003 |publisher=[[Institution of Engineering and Technology]] |isbn=9780852967102 |pages=1-2 |url=https://books.google.com/books?id=7XuU8T3ooOAC&pg=PA1}}</ref><ref name="Lea">{{cite book |last1=Lea |first1=William |title=Video on demand: Research Paper 94/68 |date=1994 |publisher=[[House of Commons Library]] |location=9 May 1994 |url=https://researchbriefings.parliament.uk/ResearchBriefing/Summary/RP94-68 |accessdate=20 September 2019}}</ref> Digital video was introduced in the 1970s,<ref name="Ghanbari"/> initially using uncompressed [[pulse-code modulation]] (PCM) requiring high [[bitrate]]s around 45{{ndash}}200 [[Mbps]] for [[standard-definition]] (SD) video,<ref name="Ghanbari"/><ref name="Lea"/> which was up to 2,000 times greater than the [[telecommunication]] [[Bandwidth (computing)|bandwidth]] (up to 100{{nbsp}}[[kilobits per second|kbps]]) available until the 1990s.<ref name="Lea"/> Similarly, uncompressed [[high-definition video|high-definition]] (HD) [[1080p]] video requires bitrates exceeding 1{{nbsp}}[[Gbps]], significantly greater than the bandwidth available in the 2000s.<ref>{{cite book |last1=Lee |first1=Jack |title=Scalable Continuous Media Streaming Systems: Architecture, Design, Analysis and Implementation |date=2005 |publisher=[[John Wiley & Sons]] |isbn=9780470857649 |page=25 |url=https://books.google.com/books?id=7fuvu52cyNEC&pg=PA25}}</ref>


Practical video coding was made possible with the [[discrete cosine transform]] (DCT).<ref name="Ghanbari"/> It was a [[lossy compression]] algorithm first proposed by [[N. Ahmed|Nasir Ahmed]], who initially applied it to [[image compression]], while he was working at [[Kansas State University]] in 1972, before it was developed into a practical compression algorithm by Ahmed with T. Natarajan and [[K. R. Rao]] at the [[University of Texas]] in 1973.<ref name="Ahmed">{{cite journal |last=Ahmed |first=Nasir |author-link=N. Ahmed |title=How I Came Up With the Discrete Cosine Transform |journal=[[Digital Signal Processing (journal)|Digital Signal Processing]] |date=January 1991 |volume=1 |issue=1 |pages=4–5 |doi=10.1016/1051-2004(91)90086-Z |url=https://www.scribd.com/doc/52879771/DCT-History-How-I-Came-Up-with-the-Discrete-Cosine-Transform}}</ref><ref name="pubDCT">{{Citation |first=Nasir |last=Ahmed |author1-link=N. Ahmed |first2=T. |last2=Natarajan |first3=K. R. |last3=Rao |title=Discrete Cosine Transform |journal=IEEE Transactions on Computers |date=January 1974 |volume=C-23 |issue=1 |pages=90–93 |doi=10.1109/T-C.1974.223784}}</ref><ref name="pubRaoYip">{{Citation |last1=Rao |first1=K. R. |authorlink1=K. R. Rao |last2=Yip |first2=P. |title=Discrete Cosine Transform: Algorithms, Advantages, Applications |publisher=Academic Press |location=Boston |year=1990 |isbn=978-0-12-580203-1}}</ref> [[Motion compensation|Motion-compensated]] DCT coding would later become the standard for video compression since the late 1980s.<ref name="Ghanbari"/><ref name="Li">{{cite book |last1=Li |first1=Jian Ping |title=Proceedings of the International Computer Conference 2006 on Wavelet Active Media Technology and Information Processing: Chongqing, China, 29-31 August 2006 |date=2006 |publisher=[[World Scientific]] |isbn=9789812709998 |page=847 |url=https://books.google.com/books?id=FZiK3zXdK7sC&pg=PA847}}</ref>
Practical video coding was made possible with the [[discrete cosine transform]] (DCT).<ref name="Ghanbari"/> It was a [[lossy compression|lossy]] [[transform coding]] technique first proposed by [[N. Ahmed|Nasir Ahmed]], who initially applied it to [[image compression]], while he was working at [[Kansas State University]] in 1972, before it was developed into a practical compression algorithm by Ahmed with T. Natarajan and [[K. R. Rao]] at the [[University of Texas]] in 1973.<ref name="Ahmed">{{cite journal |last=Ahmed |first=Nasir |author-link=N. Ahmed |title=How I Came Up With the Discrete Cosine Transform |journal=[[Digital Signal Processing (journal)|Digital Signal Processing]] |date=January 1991 |volume=1 |issue=1 |pages=4–5 |doi=10.1016/1051-2004(91)90086-Z |url=https://www.scribd.com/doc/52879771/DCT-History-How-I-Came-Up-with-the-Discrete-Cosine-Transform}}</ref><ref name="pubDCT">{{Citation |first=Nasir |last=Ahmed |author1-link=N. Ahmed |first2=T. |last2=Natarajan |first3=K. R. |last3=Rao |title=Discrete Cosine Transform |journal=IEEE Transactions on Computers |date=January 1974 |volume=C-23 |issue=1 |pages=90–93 |doi=10.1109/T-C.1974.223784}}</ref><ref name="pubRaoYip">{{Citation |last1=Rao |first1=K. R. |authorlink1=K. R. Rao |last2=Yip |first2=P. |title=Discrete Cosine Transform: Algorithms, Advantages, Applications |publisher=Academic Press |location=Boston |year=1990 |isbn=978-0-12-580203-1}}</ref>


Another key development was [[motion compensation|motion-compensated]] hybrid coding. In 1974, Ali Habibi at the [[University of Southern California]] introduced hybrid coding, which combines predictive coding with transform coding, with his algorithm initially limited to the [[spatial dimension]].<ref name="ITU"/><ref>{{cite book |last1=Ohm |first1=Jens-Rainer |title=Multimedia Signal Coding and Transmission |date=2015 |publisher=Springer |isbn=9783662466919 |pages=364 |url=https://books.google.com/books?id=e7xnBwAAQBAJ&pg=PA364}}</ref><ref>{{cite journal |last1=Habibi |first1=Ali |title=Hybrid Coding of Pictorial Data |journal=IEEE Transactions on Communications |date=1974 |volume=22 |issue=5 |pages=614–624 |doi=10.1109/TCOM.1974.1092258}}</ref> This led to the development of motion-compensated hybrid coding in 1975, when J.A. Roese and G.S. Robinson extended Habibi's hybrid coding algorithm to the [[temporal dimension]], using transform coding in the spatial dimension and predictive coding in the temporal dimension.<ref name="ITU"/> In 1981, [[Compression Labs, Inc.|Compression Labs]] founder Wen-Hsiung Chen combined motion-compensated hybrid video coding with DCT image compression into the first practical video compression algorithm, called motion-compensated DCT or adaptive scene coding.<ref name="ITU"/> Motion-compensated DCT would later become the coding standard for video compression since the late 1980s.<ref name="Ghanbari"/><ref name="Li">{{cite book |last1=Li |first1=Jian Ping |title=Proceedings of the International Computer Conference 2006 on Wavelet Active Media Technology and Information Processing: Chongqing, China, 29-31 August 2006 |date=2006 |publisher=[[World Scientific]] |isbn=9789812709998 |page=847 |url=https://books.google.com/books?id=FZiK3zXdK7sC&pg=PA847}}</ref>
⚫
The first digital video coding standard was [[H.120]], created by the [[ITU-T|CCITT]] (now ITU-T) in 1984.<ref name="history">{{cite web |title=The History of Video File Formats Infographic |url=http://www.real.com/resources/digital-video-file-formats/ |website=[[RealNetworks]] |accessdate=5 August 2019 |date=22 April 2012}}</ref> H.120 was not usable in practice, as its performance was too poor.<ref name="history" /> H.120 was based on [[differential pulse-code modulation]] (DPCM), a lossless compression algorithm that was inefficient for video coding. During the late 1980s, a number of companies began experimenting with DCT, a much more efficient form of compression for video coding. The CCITT received 14 proposals for DCT-based video compression formats, in contrast to a single proposal based on [[vector quantization]] (VQ) compression. The [[H.261]] standard was developed based on motion-compensated DCT compression.<ref name="Ghanbari"/><ref name="Li"/> H.261 was the first practical video coding standard,<ref name="history" /> and was developed using [[patents]] from a number of companies, including [[Hitachi]], [[PictureTel]], [[Nippon Telegraph and Telephone|NTT]], [[BT plc|BT]], and [[Toshiba]], among others.<ref name="h261-patents"/> Since H.261, motion-compensated DCT compression has been adopted by all the major video coding standards (including the [[H.26x]] and [[MPEG]] formats) that followed.<ref name="Ghanbari"/><ref name="Li"/>

===Video coding standards===
⚫
The first digital video coding standard was [[H.120]], created by the [[ITU-T|CCITT]] (now ITU-T) in 1984.<ref name="history">{{cite web |title=The History of Video File Formats Infographic |url=http://www.real.com/resources/digital-video-file-formats/ |website=[[RealNetworks]] |accessdate=5 August 2019 |date=22 April 2012}}</ref> H.120 was not usable in practice, as its performance was too poor.<ref name="history" /> H.120 used motion-compensated DPCM coding,<ref name="ITU"/> a lossless compression algorithm that was inefficient for video coding.<ref name="Ghanbari"/> During the late 1980s, a number of companies began experimenting with DCT, a much more efficient form of compression for video coding. The CCITT received 14 proposals for DCT-based video compression formats, in contrast to a single proposal based on [[vector quantization]] (VQ) compression. The [[H.261]] standard was developed based on motion-compensated DCT compression.<ref name="Ghanbari"/><ref name="Li"/> H.261 was the first practical video coding standard,<ref name="history" /> and was developed using [[patents]] from a number of companies, including [[Hitachi]], [[PictureTel]], [[Nippon Telegraph and Telephone|NTT]], [[BT plc|BT]], and [[Toshiba]], among others.<ref name="h261-patents"/> Since H.261, motion-compensated DCT compression has been adopted by all the major video coding standards (including the [[H.26x]] and [[MPEG]] formats) that followed.<ref name="Ghanbari"/><ref name="Li"/>


[[MPEG-1]], developed by the [[Motion Picture Experts Group]] (MPEG), followed in 1991, and it was designed to compress [[VHS]]-quality video.<ref name="history"/> It was succeeded in 1994 by [[MPEG-2]]/[[H.262/MPEG-2 Part 2|H.262]],<ref name="history"/> which was developed using patents from a number of companies, primarily [[Sony]], [[Technicolor SA|Thomson]] and [[Mitsubishi Electric]].<ref name="mp2-patents"/> MPEG-2 became the standard video format for [[DVD]] and [[Standard definition|SD]] [[digital television]].<ref name="history"/> Its motion-compensated DCT algorithm was able to achieve a [[compression ratio]] of up to 100:1, enabling the development of [[digital media]] technologies such as [[video-on-demand]] (VOD)<ref name="Lea"/> and [[high-definition television]] (HDTV).<ref name="Shishikui">{{cite journal |last1=Shishikui |first1=Yoshiaki |last2=Nakanishi |first2=Hiroshi |last3=Imaizumi |first3=Hiroyuki |title=An HDTV Coding Scheme using Adaptive-Dimension DCT |journal=Signal Processing of HDTV: Proceedings of the International Workshop on HDTV '93, Ottawa, Canada |date=October 26–28, 1993 |pages=611–618 |doi=10.1016/B978-0-444-81844-7.50072-3 |url=https://books.google.com/books?id=j9XSBQAAQBAJ&pg=PA611 |publisher=[[Elsevier]] |isbn=9781483298511}}</ref> In 1999, it was followed by [[MPEG-4 Visual|MPEG-4]]/[[H.263]], which was a major leap forward for video compression technology.<ref name="history"/> It was developed using patents from a number of companies, primarily Mitsubishi, [[Hitachi]] and [[Panasonic]].<ref name="mp4-patents"/>
[[MPEG-1]], developed by the [[Motion Picture Experts Group]] (MPEG), followed in 1991, and it was designed to compress [[VHS]]-quality video.<ref name="history"/> It was succeeded in 1994 by [[MPEG-2]]/[[H.262/MPEG-2 Part 2|H.262]],<ref name="history"/> which was developed using patents from a number of companies, primarily [[Sony]], [[Technicolor SA|Thomson]] and [[Mitsubishi Electric]].<ref name="mp2-patents"/> MPEG-2 became the standard video format for [[DVD]] and [[Standard definition|SD]] [[digital television]].<ref name="history"/> Its motion-compensated DCT algorithm was able to achieve a [[compression ratio]] of up to 100:1, enabling the development of [[digital media]] technologies such as [[video-on-demand]] (VOD)<ref name="Lea"/> and [[high-definition television]] (HDTV).<ref name="Shishikui">{{cite journal |last1=Shishikui |first1=Yoshiaki |last2=Nakanishi |first2=Hiroshi |last3=Imaizumi |first3=Hiroyuki |title=An HDTV Coding Scheme using Adaptive-Dimension DCT |journal=Signal Processing of HDTV: Proceedings of the International Workshop on HDTV '93, Ottawa, Canada |date=October 26–28, 1993 |pages=611–618 |doi=10.1016/B978-0-444-81844-7.50072-3 |url=https://books.google.com/books?id=j9XSBQAAQBAJ&pg=PA611 |publisher=[[Elsevier]] |isbn=9781483298511}}</ref> In 1999, it was followed by [[MPEG-4 Visual|MPEG-4]]/[[H.263]], which was a major leap forward for video compression technology.<ref name="history"/> It was developed using patents from a number of companies, primarily Mitsubishi, [[Hitachi]] and [[Panasonic]].<ref name="mp4-patents"/>

Revision as of 11:31, 3 November 2019

A video coding format[1][2] (or sometimes video compression format) is a content representation format for storage or transmission of digital video content (such as in a data file or bitstream). Examples of video coding formats include H.262 (MPEG-2 Part 2), MPEG-4 Part 2, H.264 (MPEG-4 Part 10), HEVC (H.265), Theora, RealVideo RV40, VP9, and AV1. A specific software or hardware implementation capable of video compression and/or decompression to/from a specific video coding format is called a video codec; an example of a video codec is Xvid, which is one of several different codecs which implements encoding and decoding videos in the MPEG-4 Part 2 video coding format in software.

Some video coding formats are documented by a detailed technical specification document known as a video coding specification. Some such specifications are written and approved by standardization organizations as technical standards, and are thus known as a video coding standard. The term 'standard' is also sometimes used for de facto standards as well as formal standards.

Video content encoded using a particular video coding format is normally bundled with an audio stream (encoded using an audio coding format) inside a multimedia container format such as AVI, MP4, FLV, RealMedia, or Matroska. As such, the user normally doesn't have a H.264 file, but instead has a .mp4 video file, which is an MP4 container containing H.264-encoded video, normally alongside AAC-encoded audio. Multimedia container formats can contain any one of a number of different video coding formats; for example the MP4 container format can contain video in either the MPEG-2 Part 2 or the H.264 video coding format, among others. Another example is the initial specification for the file type WebM, which specified the container format (Matroska), but also exactly which video (VP8) and audio (Vorbis) compression format is used inside the Matroska container, even though the Matroska container format itself is capable of containing other video coding formats (VP9 video and Opus audio support was later added to the WebM specification).

Distinction between "format" and "codec"

Although video coding formats such as H.264 are sometimes referred to as codecs, there is a clear conceptual difference between a specification and its implementations. Video coding formats are described in specifications, and software or hardware to encode/decode data in a given video coding format from/to uncompressed video are implementations of those specifications. As an analogy, the video coding format H.264 (specification) is to the codec OpenH264 (specific implementation) what the C Programming Language (specification) is to the compiler GCC (specific implementation). Note that for each specification (e.g. H.264), there can be many codecs implementing that specification (e.g. x264, OpenH264, H.264/MPEG-4 AVC products and implementations).

This distinction is not consistently reflected terminologically in the literature. The H.264 specification calls H.261, H.262, H.263, and H.264 video coding standards and does not contain the word codec.[3] The Alliance for Open Media clearly distinguishes between the AV1 video coding format and the accompanying codec they are developing, but calls the video coding format itself a video codec specification.[4] The VP9 specification calls the video coding format VP9 itself a codec.[5]

As an example of conflation, Chromium's[6] and Mozilla's[7] pages listing their video format support both call video coding formats such as H.264 codecs. As another example, in Cisco's announcement of a free-as-in-beer video codec, the press release refers to the H.264 video coding format as a "codec" ("choice of a common video codec"), but calls Cisco's implementation of a H.264 encoder/decoder a "codec" shortly thereafter ("open-source our H.264 codec").[8]

A video coding format does not dictate all algorithms used by a codec implementing the format. For example, a large part of how video compression typically works is by finding similarities between video frames (block-matching), and then achieving compression by copying previously-coded similar subimages (e.g., macroblocks) and adding small differences when necessary. Finding optimal combinations of such predictors and differences is an NP-hard problem,[9] meaning that it is practically impossible to find an optimal solution. While the video coding format must support such compression across frames in the bitstream format, by not needlessly mandating specific algorithms for finding such block-matches and other encoding steps, the codecs implementing the video coding specification have some freedom to optimize and innovate in their choice of algorithms. For example, section 0.5 of the H.264 specification says that encoding algorithms are not part of the specification.[3] Free choice of algorithm also allows different space–time complexity trade-offs for the same video coding format, so a live feed can use a fast but space-inefficient algorithm, while a one-time DVD encoding for later mass production can trade long encoding-time for space-efficient encoding.

History

Video compression

The concept of analog video compression dates back to 1929, when R.D. Kell in Britain proposed the concept of transmitting only the portions of the scene that changed from frame-to-frame. The concept of digital video compression dates back to 1952, when Bell Labs researchers B.M. Oliver and C.W. Harrison proposed the use of differential pulse-code modulation (DPCM) in video coding. The concept of motion compensation dates back to 1959, when NHK researchers Y. Taki, M. Hatori and S. Tanaka proposed inter-frame video coding.[10] In 1967, University of London researchers A.H. Robinson and C. Cherry proposed run-length encoding (RLE), a lossless compression scheme, to slightly reduce the transmission bandwidth of analog television signals.[11]

The earliest digital video coding algorithms were either for uncompressed video or used lossless compression, both methods inefficient and impractical for digital video coding.[12][13] Digital video was introduced in the 1970s,[12] initially using uncompressed pulse-code modulation (PCM) requiring high bitrates around 45–200 Mbps for standard-definition (SD) video,[12][13] which was up to 2,000 times greater than the telecommunication bandwidth (up to 100 kbps) available until the 1990s.[13] Similarly, uncompressed high-definition (HD) 1080p video requires bitrates exceeding 1 Gbps, significantly greater than the bandwidth available in the 2000s.[14]

Practical video coding was made possible with the discrete cosine transform (DCT).[12] It was a lossy transform coding technique first proposed by Nasir Ahmed, who initially applied it to image compression, while he was working at Kansas State University in 1972, before it was developed into a practical compression algorithm by Ahmed with T. Natarajan and K. R. Rao at the University of Texas in 1973.[15][16][17]

Another key development was motion-compensated hybrid coding. In 1974, Ali Habibi at the University of Southern California introduced hybrid coding, which combines predictive coding with transform coding, with his algorithm initially limited to the spatial dimension.[10][18][19] This led to the development of motion-compensated hybrid coding in 1975, when J.A. Roese and G.S. Robinson extended Habibi's hybrid coding algorithm to the temporal dimension, using transform coding in the spatial dimension and predictive coding in the temporal dimension.[10] In 1981, Compression Labs founder Wen-Hsiung Chen combined motion-compensated hybrid video coding with DCT image compression into the first practical video compression algorithm, called motion-compensated DCT or adaptive scene coding.[10] Motion-compensated DCT would later become the coding standard for video compression since the late 1980s.[12][20]

Video coding standards

The first digital video coding standard was H.120, created by the CCITT (now ITU-T) in 1984.[21] H.120 was not usable in practice, as its performance was too poor.[21] H.120 used motion-compensated DPCM coding,[10] a lossless compression algorithm that was inefficient for video coding.[12] During the late 1980s, a number of companies began experimenting with DCT, a much more efficient form of compression for video coding. The CCITT received 14 proposals for DCT-based video compression formats, in contrast to a single proposal based on vector quantization (VQ) compression. The H.261 standard was developed based on motion-compensated DCT compression.[12][20] H.261 was the first practical video coding standard,[21] and was developed using patents from a number of companies, including Hitachi, PictureTel, NTT, BT, and Toshiba, among others.[22] Since H.261, motion-compensated DCT compression has been adopted by all the major video coding standards (including the H.26x and MPEG formats) that followed.[12][20]

MPEG-1, developed by the Motion Picture Experts Group (MPEG), followed in 1991, and it was designed to compress VHS-quality video.[21] It was succeeded in 1994 by MPEG-2/H.262,[21] which was developed using patents from a number of companies, primarily Sony, Thomson and Mitsubishi Electric.[23] MPEG-2 became the standard video format for DVD and SD digital television.[21] Its motion-compensated DCT algorithm was able to achieve a compression ratio of up to 100:1, enabling the development of digital media technologies such as video-on-demand (VOD)[13] and high-definition television (HDTV).[24] In 1999, it was followed by MPEG-4/H.263, which was a major leap forward for video compression technology.[21] It was developed using patents from a number of companies, primarily Mitsubishi, Hitachi and Panasonic.[25]

Perhaps the most widely used video coding format as of 2016 is H.264/MPEG-4 AVC. It was developed in 2003 using patents from a number of organizations, primarily Panasonic, Godo Kaisha IP Bridge and LG Electronics.[26] H.264 is one of the video encoding standards for Blu-ray Discs; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from YouTube, Netflix, Vimeo, and the iTunes Store, web software such as the Adobe Flash Player and Microsoft Silverlight, and also various HDTV broadcasts over terrestrial (Advanced Television Systems Committee standards, ISDB-T, DVB-T or DVB-T2), cable (DVB-C), and satellite (DVB-S2).

A main problem for many video coding formats has been patents, making it expensive to use and/or potentially risking a patent lawsuit due to submarine patents. The motivation behind many recently designed video coding formats such as Theora, VP8, and VP9 have been to create a (libre) video coding standard covered only by royalty-free patents.[27] Patent status has also been a major point of contention for the choice of which video formats the mainstream web browsers will support inside the HTML5 video tag.

The next generation video coding format is HEVC (H.265). It is heavily patented, with the majority of patents belonging to Samsung Electronics, GE, NTT and JVC Kenwood.[28] It is currently being challenged by the aiming-to-be-freely-licensed AV1 format.

Timeline of international video compression standards
Year Standard Publisher(s) Committee(s) Patent holders Popular implementations
1984 H.120 ITU-T VCEG —
1988 H.261 ITU-T VCEG Hitachi, PictureTel, NTT, BT, Toshiba, etc.[22] Videoconferencing, videotelephony
1992 Motion JPEG (MJPEG) JPEG JPEG — QuickTime
1993 MPEG-1 Part 2 ISO, IEC MPEG Fujitsu, IBM, Matsushita, etc.[29] Video-CD
1995 H.262/MPEG-2 Part 2 ISO, IEC, ITU-T MPEG, VCEG Sony, Thomson, Mitsubishi, etc.[23] DVD Video, Blu-ray, DVB, ATSC, SVCD, SDTV
1996 H.263 ITU-T VCEG Mitsubishi, Hitachi, Panasonic, etc.[25] Videoconferencing, videotelephony, video on mobile phones (3GP)
1999 MPEG-4 Part 2 ISO, IEC MPEG Mitsubishi, Hitachi, Panasonic, etc.[25] Video on Internet (DivX, Xvid ...)
2001 Motion JPEG 2000 (MJ2) JPEG[30][31] JPEG — Digital cinema[32]
2003 H.264/MPEG-4 AVC ISO, IEC, ITU-T MPEG, VCEG Panasonic, Godo Kaisha IP Bridge, LG, etc.[26] Blu-ray, HD DVD, DVB, ATSC, HDTV, iPod Video, Apple TV, videoconferencing, VOD
2006 VC-1 SMPTE SMPTE Microsoft, Panasonic, LG, Samsung, etc.[33] Blu-ray, video on Internet
2013 H.265/MPEG-H HEVC ISO, IEC, ITU-T MPEG, VCEG Samsung, GE, NTT, JVC Kenwood, etc.[28][34] Ultra HD Blu-ray, DVB, ATSC 3.0, UHD streaming, High Efficiency Image Format, macOS High Sierra, iOS 11
2018 AV1 AOMedia AOMedia — HTML5 video

Lossless, lossy, and uncompressed video coding formats

Consumer video is generally compressed using lossy video codecs, since that results in significantly smaller files than lossless compression. While there are video coding formats designed explicitly for either lossy or lossless compression, some video coding formats such as Dirac and H.264 support both.

Uncompressed video formats, such as Clean HDMI, is a form of lossless video used in some circumstances such as when sending video to a display over a HDMI connection. Some high-end cameras can also capture video directly in this format.

Intra-frame video coding formats

Interframe compression complicates editing of an encoded video sequence.[35] One subclass of relatively simple video coding formats are the intra-frame video formats, such as DV, in which each frame of the video stream is compressed independently without referring to other frames in the stream, and no attempt is made to take advantage of correlations between successive pictures over time for better compression. One example is Motion JPEG, which is simply a sequence of individually JPEG-compressed images. This approach is quick and simple, at the expense the encoded video being much larger than a video coding format supporting Inter frame coding.

Because interframe compression copies data from one frame to another, if the original frame is simply cut out (or lost in transmission), the following frames cannot be reconstructed properly. Making 'cuts' in intraframe-compressed video while video editing is almost as easy as editing uncompressed video: one finds the beginning and ending of each frame, and simply copies bit-for-bit each frame that one wants to keep, and discards the frames one doesn't want. Another difference between intraframe and interframe compression is that, with intraframe systems, each frame uses a similar amount of data. In most interframe systems, certain frames (such as "I frames" in MPEG-2) aren't allowed to copy data from other frames, so they require much more data than other frames nearby.[36]

It is possible to build a computer-based video editor that spots problems caused when I frames are edited out while other frames need them. This has allowed newer formats like HDV to be used for editing. However, this process demands a lot more computing power than editing intraframe compressed video with the same picture quality.

Profiles and levels

A video coding format can define optional restrictions to encoded video, called profiles and levels. It is possible to have a decoder which only supports decoding a subset of profiles and levels of a given video format, for example to make the decoder program/hardware smaller, simpler, or faster.

A profile restricts which encoding techniques are allowed. For example, the H.264 format includes the profiles baseline, main and high (and others). While P-slices (which can be predicted based on preceding slices) are supported in all profiles, B-slices (which can be predicted based on both preceding and following slices) are supported in the main and high profiles but not in baseline.[37]

A level is a restriction on parameters such as maximum resolution and data rates.[37]

See also

References and notes

  1. ^ The term "video coding" can he seen in e.g. the names Advanced Video Coding, High Efficiency Video Coding, and Video Coding Experts Group
  2. ^ Thomas Wiegand; Gary J. Sullivan; Gisle Bjontegaard; Ajay Luthra (July 2003). "Overview of the H.264 / AVC Video Coding Standard" (PDF). IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY. {{cite web}}: Unknown parameter |last-author-amp= ignored (|name-list-style= suggested) (help)
  3. ^ a b "SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS : Infrastructure of audiovisual services – Coding of moving video : Advanced video coding for generic audiovisual services". Itu.int. Retrieved 6 January 2015.
  4. ^ "Front Page". Alliance for Open Media. Retrieved 2016-05-23.
  5. ^ Adrian Grange; Peter de Rivaz; Jonathan Hunt. "VP9 Bitstream & Decoding Process Specification" (PDF). Google. {{cite web}}: Unknown parameter |last-author-amp= ignored (|name-list-style= suggested) (help)
  6. ^ "Audio/Video". The Chromium Projects. Retrieved 2016-05-23.
  7. ^ "Media formats supported by the HTML audio and video elements". Mozilla. Retrieved 2016-05-23.
  8. ^ Rowan Trollope (2013-10-30). "Open-Sourced H.264 Removes Barriers to WebRTC". Cisco. Retrieved 2016-05-23.
  9. ^ "Chapter 3 : Modified A* Prune Algorithm for finding K-MCSP in video compression" (PDF). Shodhganga.inflibnet.ac.in. Retrieved 2015-01-06.
  10. ^ a b c d e "History of Video Compression". ITU-T. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6). July 2002. pp. 11, 24–9, 33, 53–5. Retrieved 3 November 2019.
  11. ^ Robinson, A. H.; Cherry, C. (1967). "Results of a prototype television bandwidth compression scheme". Proceedings of the IEEE. 55 (3). IEEE: 356–364. doi:10.1109/PROC.1967.5493.
  12. ^ a b c d e f g h Ghanbari, Mohammed (2003). Standard Codecs: Image Compression to Advanced Video Coding. Institution of Engineering and Technology. pp. 1–2. ISBN 9780852967102.
  13. ^ a b c d Lea, William (1994). Video on demand: Research Paper 94/68. 9 May 1994: House of Commons Library. Retrieved 20 September 2019.{{cite book}}: CS1 maint: location (link)
  14. ^ Lee, Jack (2005). Scalable Continuous Media Streaming Systems: Architecture, Design, Analysis and Implementation. John Wiley & Sons. p. 25. ISBN 9780470857649.
  15. ^ Ahmed, Nasir (January 1991). "How I Came Up With the Discrete Cosine Transform". Digital Signal Processing. 1 (1): 4–5. doi:10.1016/1051-2004(91)90086-Z.
  16. ^ Ahmed, Nasir; Natarajan, T.; Rao, K. R. (January 1974), "Discrete Cosine Transform", IEEE Transactions on Computers, C-23 (1): 90–93, doi:10.1109/T-C.1974.223784
  17. ^ Rao, K. R.; Yip, P. (1990), Discrete Cosine Transform: Algorithms, Advantages, Applications, Boston: Academic Press, ISBN 978-0-12-580203-1
  18. ^ Ohm, Jens-Rainer (2015). Multimedia Signal Coding and Transmission. Springer. p. 364. ISBN 9783662466919.
  19. ^ Habibi, Ali (1974). "Hybrid Coding of Pictorial Data". IEEE Transactions on Communications. 22 (5): 614–624. doi:10.1109/TCOM.1974.1092258.
  20. ^ a b c Li, Jian Ping (2006). Proceedings of the International Computer Conference 2006 on Wavelet Active Media Technology and Information Processing: Chongqing, China, 29-31 August 2006. World Scientific. p. 847. ISBN 9789812709998.
  21. ^ a b c d e f g "The History of Video File Formats Infographic". RealNetworks. 22 April 2012. Retrieved 5 August 2019.
  22. ^ a b "ITU-T Recommendation declared patent(s)". ITU. Retrieved 12 July 2019.
  23. ^ a b "MPEG-2 Patent List" (PDF). MPEG LA. Retrieved 7 July 2019.
  24. ^ Shishikui, Yoshiaki; Nakanishi, Hiroshi; Imaizumi, Hiroyuki (October 26–28, 1993). "An HDTV Coding Scheme using Adaptive-Dimension DCT". Signal Processing of HDTV: Proceedings of the International Workshop on HDTV '93, Ottawa, Canada. Elsevier: 611–618. doi:10.1016/B978-0-444-81844-7.50072-3. ISBN 9781483298511.
  25. ^ a b c "MPEG-4 Visual - Patent List" (PDF). MPEG LA. Retrieved 6 July 2019.
  26. ^ a b "AVC/H.264 – Patent List" (PDF). MPEG LA. Retrieved 6 July 2019.
  27. ^ https://blogs.cisco.com/collaboration/world-meet-thor-a-project-to-hammer-out-a-royalty-free-video-codec
  28. ^ a b "HEVC Patent List" (PDF). MPEG LA. Retrieved 6 July 2019.
  29. ^ "ISO Standards and Patents". ISO. Retrieved 10 July 2019.
  30. ^ "Motion JPEG 2000 Part 3". Joint Photographic Experts Group, JPEG, and Joint Bi-level Image experts Group, JBIG. Archived from the original on 22 September 2012. Retrieved 21 June 2014.
  31. ^ Taubman, David; Marcellin, Michael (2012). JPEG2000 Image Compression Fundamentals, Standards and Practice: Image Compression Fundamentals, Standards and Practice. Springer Science & Business Media. ISBN 9781461507994.
  32. ^ Swartz, Charles S. (2005). Understanding Digital Cinema: A Professional Handbook. Taylor & Francis. p. 147. ISBN 9780240806174.
  33. ^ "VC-1 Patent List" (PDF). MPEG LA. Retrieved 11 July 2019.
  34. ^ "HEVC Advance Patent List". HEVC Advance. Retrieved 6 July 2019.
  35. ^ Bhojani, D.R. "4.1 Video Compression" (PDF). Hypothesis. Retrieved 6 March 2013.
  36. ^ Jaiswal, R.C. (2009). Audio-Video Engineering. Pune, Maharashtra: Nirali Prakashan. p. 3.55. ISBN 9788190639675.
  37. ^ a b Jan Ozer. "Encoding options for H.264 video". Adobe.com. Retrieved 6 January 2015.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Video_coding_format&oldid=924361148"