Video codecs
There are many video codecs existing and in use in the real world.
Some are more widespread and well known as others, others are less common or can even be
considered "exotic".
Lossy vs lossless vs uncompressed?
Fear, Uncertainty and Doubt (FUD)
First of all, I'd like to note that there is currently some confusion in the video domain, regarding
the naming and properties of artefacts introduced by storing video in digtial form.
As one example, on the product site of Blackmagic's Decklink video cards
they were talking about "unrivaled video quality", but wrongly equating
"compressed" with "lossy". They also showed an image suggesting uncompressed as opposite of subsampling.
Here is a screenshot from December 2013.
Even though, Blackmagic could be considered to be addressing only semi-pro users, it still
shows that even well-known vendors in the video domain sometimes unfortunately may communicate
erroneous and/or confusing information.
Archivists might want to keep this in mind when evaluating which products or solutions to use
for their purposes.
Filesizes
Digital video produces vast amounts of data. Take PAL-SD material (only image, no audio),
stored as YUV422 with 8-bits per component (=16-bits per pixel) and a resolution of 720x576
pixels, for example.
Applying no compression at all (=uncompressed) results in:
720 x 576 x 16 bits x 25 frames ≥ 1.159 GiB/Minute
For YUV422, 10 bpc (10+5+5 = 20 bits per pixel), this would be:
720 x 576 x 20 bits x 25 frames ≥ 1.448 GiB/Minute
So you would fit no more than 4 minutes of video on a whole 4.7 GiB DVD.
Since these values are already uncomfortable (even for 422-subsampled PAL), imagine HD and beyond.
Therefore, you will hardly find uncompressed video in the wild, except for temporary editing use cases.
Currently, most video is stored "lossy":
Even most digital cameras
already use some form of compression before saving their data.
The majority of video codecs currently in use, perform a
so called "lossy compression".
As the name implies, data is lost.
Currently, this is even the normal case for professional video production, broadcasting,
digital cinemas, etc.
Examples of lossy compression used for professional production are:
Even for archiving, it's currently still common to see people using lossy video compression.
Maybe because many archives follow the example of large and well-known broadcasting archives –
sometimes overlooking that the latter's purpose focuses on the needs of production rather than longterm
preservation.
Extending the above list, here are additional formats, usually more common on the consumer
market, which are used by some to store their videos in:
All of the aforementioned video codecs are lossy and should not be used for archiving,
unless this is already the format the original source was in.
The technical guidelines by IASA, for archiving video (TC-06), are still work in progress, but
in the IASA guidelines for preserving audio (TC-03), chapter 11 (p.8), it says quite clearly:
"[...] formats employing data reduction [...] based on perceptual coding (“lossy codecs”) must not be used.
Transfers employing such data reduction result in the irretrievable loss of parts of the primary information. The results of such “lossy” data reduction may sound identical or very similar to the unreduced (linear) signal, but the further use of the data reduced signal will be severely restricted.
These archival principles should also be applied, whenever possible, to the creation of original recordings made with the intention of being archived."
In the online version of the TC03 paragraph "11. Data reduction",
an additional comment clarifies the reason for this even further:
"Its use is, however, counter to the ethical principle of preserving as much of the primary information as possible. Data reduction does not permit the restoration of the signal to its original acoustic condition and will, in addition, limit the further use of the recording because of the artefacts generated when cascading perceptually coded material - for example, in the making of a new programme incorporating the original [recordings]."
So, for long-term preservation of video, lossy is not an option.
As we've seen above, storing video "uncompressed" is currently still non-trivial to
handle, regarding its filesize.
See details about "uncompressed" for video, below.
Interesting for archiving: "lossless compression".
Uncompressed is not the only option for avoiding loss of image information when
storing digital video.
There are compression algorithms which reduce the filesize, but preserve every bit
in its original state, even after recompressing an infinite number of times.
This is called "lossless compression".
For generic data, most people use lossless compression in their daily routine when creating a
"Zip file (.zip)".
There are a few codecs providing lossless compression for video.
In practice however, most of them are unsuitable for long-term archiving for several reasons.
So, although this list (see below) may be longer than expected, it boils down to
only a handful of potential candidates for long-term preservation.
For example, the "MSU Graphics & Media Lab" of the department of computer science of Lomonosov Moscow State University published
test results for different lossless codecs in 2007.
They have performed comparisons of the following lossless codecs:
- Alpary
- ArithYuv
- AVIzlib
- CamStudio GZIP
- CorePNG
- FastCodec
- FFV1
- Huffyuv
- Lagarith
- LOCO
- LZO
- MSU Lab
- PICVideo
- Snow
- x264
- YULS
NOTE:
It might be interesting to point out that JPEG2000-lossless is missing in this list,
although the same group has published a JPEG2000 (lossy) comparison for still-images in 2005 (2 years before).
Another one missing is BBC's Dirac codec,
but because it was released about one year after their tests.
For long-term archiving purposes, the most important factor for choosing a codec is
to be able to preserve the content adequately in a stable manner and its
vendor-independent accessibility over time.
Quoting the online page of chapter 11 of IASA's TC03,
this excludes proprietary formats:
"In the case of recordings originated in data reduced formats, a major problem with obsolescence of equipment may arise when the format of origination is of a proprietary character [...]"
Most of the lossless codecs on that list did not meet the above mentioned criteria:
Some could not preserve the input pixel-format and colorspace (e.g. YUV or RGB only),
others were limited to proprietary implementations, or even bound to certain operating systems.
Adding
Uncompressed,
JPEG2000-lossless and
Dirac, only a small number of lossless codecs remain for closer
evaluation for video archiving purposes:
- Uncompressed
- JPEG2000-lossless
- Dirac
- FFV1
- H.264-lossless (x264)
