Internet DRAFT - draft-maxwell-videocodec-requirements
draft-maxwell-videocodec-requirements
Network Working Group GM. Maxwell
Internet-Draft Xiph.Org
Intended status: Informational K. Walsh
Expires: April 18, 2013 October 15, 2012
Requirements for an Internet Video Codec
draft-maxwell-videocodec-requirements-00
Abstract
This document provides specific requirements for an Internet video
codec. These requirements address quality, bit-rate, loss
robustness, application suitability, as well as other desirable
properties.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 18, 2013.
Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Maxwell & Walsh Expires April 18, 2013 [Page 1]
Internet-Draft Video Codec Requirements October 2012
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Applications . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1. Point to point video calls . . . . . . . . . . . . . . . . 5
3.2. Video Conferencing . . . . . . . . . . . . . . . . . . . . 5
3.3. Telepresence . . . . . . . . . . . . . . . . . . . . . . . 5
3.4. Teleoperation and Remote Software Services . . . . . . . . 5
3.5. Other applications . . . . . . . . . . . . . . . . . . . . 6
4. Constraints Imposed by the Internet on the Codec . . . . . . . 7
5. Detailed Basic Requirements . . . . . . . . . . . . . . . . . 9
5.1. Quality and bit-rate . . . . . . . . . . . . . . . . . . . 9
5.2. Computational resources . . . . . . . . . . . . . . . . . 9
6. Additional considerations . . . . . . . . . . . . . . . . . . 10
7. Encoder side potential for improvement . . . . . . . . . . . . 11
8. Bit error robustness . . . . . . . . . . . . . . . . . . . . . 12
9. Legacy compatibility . . . . . . . . . . . . . . . . . . . . . 13
10. Security Considerations . . . . . . . . . . . . . . . . . . . 14
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
12. Informative References . . . . . . . . . . . . . . . . . . . . 16
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17
Maxwell & Walsh Expires April 18, 2013 [Page 2]
Internet-Draft Video Codec Requirements October 2012
1. Introduction
This document provides requirements for a video codec designed
specifically for use over the Internet. The requirements attempt to
address the needs of the most common Internet video transmission
applications and to ensure good quality when operating in conditions
that are typical for the Internet. These requirements address
quality, bit-rate, and loss robustness. Other desirable codec
properties are considered as well.
Maxwell & Walsh Expires April 18, 2013 [Page 3]
Internet-Draft Video Codec Requirements October 2012
2. Definitions
Codec bit-rates in bits per second (b/s) will be considered without
counting any overhead (IP/UDP/RTP headers, padding, ...).
Maxwell & Walsh Expires April 18, 2013 [Page 4]
Internet-Draft Video Codec Requirements October 2012
3. Applications
The following applications should be considered for Internet video
codecs, along with their requirements:
o Live video streaming
o Video on demand
o Point to point video calls
o Video Conferencing
o Telepresence
o Teleoperation
o Remote software services
o Other applications
3.1. Point to point video calls
Point to point calls are calls from two "standard" (fixed or mobile)
phones, desktop, portable computers, or tablets, and implemented in
hardware or software.
3.2. Video Conferencing
Video Conferencing applications (which support multi-party calls)
have additional requirements on top of the requirements for point-to-
point calls. Conferencing systems often have greater network
bandwidth available. The ability to vary the bit-rate (VBR) is a
desirable feature for the codec. This not only saves bandwidth "on
average", but it can also help conference servers make more efficient
use of the available bandwidth by using more bandwidth for important
video streams and less bandwidth for less important ones.
3.3. Telepresence
Most telepresence applications can be considered to be essentially
very high quality video conferencing environments, so all of the
conferencing requirements also apply to telepresence.
3.4. Teleoperation and Remote Software Services
Teleoperation applications are similar to telepresence, with the
exception that they involve remote physical interactions. For
Maxwell & Walsh Expires April 18, 2013 [Page 5]
Internet-Draft Video Codec Requirements October 2012
example, the user may be controlling a robot while receiving a real-
time video feed from that robot. The other requirements of
telepresence apply to teleoperation as well.
The requirements for remote software services are similar to those of
teleoperation. These applications include remote desktop
applications, remote virtualization, and interactive media
applications being rendered remotely (e.g. video games rendered on
central servers).
3.5. Other applications
The above list is by no means a complete list of all applications
involving interactive video transmission on the Internet. However,
it is believed that meeting the needs of all these different
applications should be sufficient to ensure that most applications
not listed will also be met.
Maxwell & Walsh Expires April 18, 2013 [Page 6]
Internet-Draft Video Codec Requirements October 2012
4. Constraints Imposed by the Internet on the Codec
The bandwidth requirements of video are a significant obstacle for
Internet deployment. A substantial portion of the hosts on the
Internet have connectivity sufficient to carry perceptually lossless
audio, even in inefficient uncompressed form. However, a much
smaller portion of hosts have connectivity sufficient for the 200+
megabits per second required for uncompressed 30fps standard
definition video, and expected resolutions for Internet video are
increasing. Even the highest resolutions widely used off the
Internet, where wide-area bandwidth is not a constraint, have not yet
reached perceptual losslessness. In addition to increases in
resolution, operating models which are less broadcast-oriented
(including video on demand and video conferencing) limit the traffic
mitigation effectiveness of CDNs and multicast, though support for
these technologies remains essential. Because there are a few
applications where increases in bandwidth efficiency are not
important and many where improved efficiency is essential--such as
delivering HD video to bandwidth-constrained network edges--it is
important that the codec deliver competitive quality per bitrate and
support a wide range of bandwidths.
Packet losses are inevitable on the Internet and dealing with them is
one of requirements for an Internet video codec. Efficient video
compression typically uses very high gain backward prediction, which
can result in infinite error propagation in the worst case if
measures are not taken to mitigate it. Error propagation is usually
mitigated in traditional file- and broadcast-oriented codecs through
key-frames, periodic intra refresh, and constrained back-reference
structure. While these techniques are important, and also enable
random access, a codec designed for the Internet should also be able
to take advantage of bidirectional communication to reduce the impact
of loss when possible.
In many high-latency and non-realtime applications, however, the
relevant transport is lossless. While random access still is
important, general error tolerance is not, and the codec may support
modes which have very low error tolerance--including ones which
prevent packet decode in the presence of loss, if it results in
efficiency gains for non-realtime applications.
For interactive applications latency is an important codec
performance metric and many common input and output devices add
frames of latency. To avoid adding further delays the codec must
support operating in a mode that adds no more delay than that from
processing a single frame at a time. Modes which permit sub-frame
encoding may be useful but are hampered by the lack of subframe
support in existing input and output devices.
Maxwell & Walsh Expires April 18, 2013 [Page 7]
Internet-Draft Video Codec Requirements October 2012
Another important property of the Internet is that it is mostly a
best-effort network, with no guaranteed bandwidth. This means that
the codec has to be able to vary its output bit-rate dynamically (in
real-time), without requiring an out-of-band signaling mechanism, and
without causing artifacts at the bit-rate change boundaries. Because
the complete range of useful bit-rates may not be achievable at a
single resolution the codec may need to support changing resolutions
on the fly. Additional desirable features are:
o Having the possibility to use smooth bit-rate changes with high
bit-rate resolution;
o Making it possible for a codec to adapt its bit-rate based on the
source signal being encoded (source-controlled VBR) to maximize
the quality for a certain _average_ bit-rate.
Because the Internet transmits data in bytes, a codec should produce
compressed data in integer numbers of bytes. In general, the codec
design should take into consideration explicit congestion
notification (ECN) and multicast and may include features that would
improve the quality of an ECN or multicast enabled deployment.
The IETF has defined a set of application-layer protocols to be used
for transmitting real-time transport of multimedia data, including
video. It is thus important for the resulting codec to be easy to
use with these protocols. For example, it must be possible to create
an [RTP] payload format that conforms to BCP 36 [PAYLOADS]. If any
codec parameters need to be negotiated between end-points, the
negotiation should be as easy as possible to carry over SIP
[RFC3261]/SDP [RFC4566] or alternatively over XMPP [RFC6120]/Jingle
[XEP-0167].
Maxwell & Walsh Expires April 18, 2013 [Page 8]
Internet-Draft Video Codec Requirements October 2012
5. Detailed Basic Requirements
This section summarizes all the constraints imposed by the target
applications and by the Internet into a set of actual requirements
for codec development.
5.1. Quality and bit-rate
The quality of a codec is directly linked to the bit-rate, so these
two must be considered jointly. When comparing the bit-rate of
codecs, the overhead of IP/UDP/RTP headers should not be considered,
but any additional bits required in the RTP payload format after the
header (e.g. required signaling) should be considered. In terms of
quality vs bit-rate, the codec to be developed must be better than
the following codecs, that are generally considered as royalty-free:
o VP8
o Theora
It is desirable for the codecs to support source-controlled variable
bit-rate (VBR) to take advantage from the fact that different inputs
require a different bitrate to achieve the same quality.
5.2. Computational resources
The resulting codec should be implementable on a wide range of
devices, and should not have a design which gratuitously complicates
low power ASIC implementations. While the codec must not depend on
special hardware features or instructions, the codec design should
allow implementations to take full advantage of hardware accelerators
and vector instructions where available. Complexity should generally
scale with resolution, and it is also desirable to support multiple
encoder and decoder complexity levels via mechanisms other than
resolution, in order to achieve the best possible bitrate/quality
trade-off available across many kinds of devices without unduly
constraining resolution. The codec should also be able to take
advantages of advances in computer speed and the deployment of
hardware accelerators which would allow the use of higher complexity
modes in a broader set of applications.
In addition to computational complexity, dynamic memory for reference
storage is a significant resource constraint for video codecs. It is
desirable that the codec support different memory usage tradeoffs to
fit on more devices, and that the codec not require implementations
to utilize more memory without reasonable efficiency gains.
Maxwell & Walsh Expires April 18, 2013 [Page 9]
Internet-Draft Video Codec Requirements October 2012
6. Additional considerations
There are additional features or characteristics that may be
desirable under some circumstances, but should not be part of the
strict requirements. The benefit of meeting these considerations
should be weighted against the associated cost.
Maxwell & Walsh Expires April 18, 2013 [Page 10]
Internet-Draft Video Codec Requirements October 2012
7. Encoder side potential for improvement
In most video codecs, it is possible to improve the quality by
improving the encoder without breaking compatibility (i.e. without
changing the decoder). Potential for improvement varies from one
codec to another. All things being equal, being able to improve a
codec after the bit-stream is a desirable property. However, this
should not be done at the expense of quality in a straight-forward
encoder.
Maxwell & Walsh Expires April 18, 2013 [Page 11]
Internet-Draft Video Codec Requirements October 2012
8. Bit error robustness
The vast majority of Internet-based applications do not need to be
robust to bit errors because packets either arrive unaltered, or do
not arrive at all. Considering that, the emphasis should be on
packet loss robustness and packet loss concealment. That being said,
it is often the case that extra robustness to bit errors can be
achieved at no cost at all (i.e. no increase in size, complexity or
bit-rate, no decrease in quality or packet loss robustness, ...). In
those cases then it is useful to make a change that increases the
robustness to bit errors. This can be useful for applications that
use UDP Lite transmission (e.g. over a wireless LAN). Robustness to
packet loss should *never* be sacrificed to achieve higher bit error
robustness.
Maxwell & Walsh Expires April 18, 2013 [Page 12]
Internet-Draft Video Codec Requirements October 2012
9. Legacy compatibility
In order to create the best possible codec for the Internet, there is
no general requirement for compatibility with legacy Internet codecs.
However, compatibility with commonly used video color formats is
desirable.
Maxwell & Walsh Expires April 18, 2013 [Page 13]
Internet-Draft Video Codec Requirements October 2012
10. Security Considerations
Although this document itself does not have security considerations,
this section describes the security requirements for the codec.
Just like for any protocol to be used over the Internet, security is
a very important aspect to consider. This goes beyond the obvious
considerations of preventing buffer overflows and similar attacks
that can lead to denial-of-service or remote code execution. One
very important security aspect is to make sure that the decoders have
a bounded and reasonable worst case complexity. This prevents an
attacker from causing a DoS by sending packets that are specially
crafted to take a very long (or infinite) time to decode.
Maxwell & Walsh Expires April 18, 2013 [Page 14]
Internet-Draft Video Codec Requirements October 2012
11. IANA Considerations
This document has no actions for IANA.
Maxwell & Walsh Expires April 18, 2013 [Page 15]
Internet-Draft Video Codec Requirements October 2012
12. Informative References
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC6120] Saint-Andre, P., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 6120, March 2011.
[XEP-0167]
Ludwig, S., Saint-Andre, P., Egan, S., McQueen, R., and D.
Cionoiu, "Jingle RTP Sessions", XSF XEP 0167,
December 2009.
[PAYLOADS]
Handley, M. and C. Perkins, "Guidelines for Writers of RTP
Payload Format Specifications", RFC 2736, BCP 36.
[RTP] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for real-time
applications", RFC 3550.
Maxwell & Walsh Expires April 18, 2013 [Page 16]
Internet-Draft Video Codec Requirements October 2012
Authors' Addresses
Gregory Maxwell
Xiph.Org
Email: greg@xiph.org
Kat Walsh
Email: kat@mindspillage.org
Maxwell & Walsh Expires April 18, 2013 [Page 17]
