Internet Draft R. Prandolini draft-prandolini-mmusic-jpip-requirements-00.txt DSTO, Australia Expires: October 2002 R. Clark Elysium, Ltd. S. Houchin Eastman Kodak April 2002 JPIP Requirements and Profiles Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes the requirements for an intelligent protocol for serving JPEG 2000 compressed image and metadata (JPIP). This document represents version 2 of the requirements and profiles of the JPIP standard (ISO/IEC 15444-9) found in document WG1N2522 of ISO/IEC JTC1/SC29/WG1 (the JPEG committee). Table of Contents 1. Introduction..................................................2 2. Scope of JPIP.................................................2 3. Markets and applications......................................4 4. Requirements..................................................4 4.1 Area of interest selection (M)............................4 4.2 Data transmission efficiency (M)..........................5 4.3 Session persistence (O)...................................5 4.4 Error handling (O)........................................5 4.5 Rate-control (O)..........................................5 Prandolini Expires - October 2002 [Page 1] JPIP Requirements and Profiles April 2002 4.6 Support for simultaneous access (O).......................5 4.7 Retrieve sub-objects from a JPEG 2000 file (M for JPX)....6 4.8 Support codestreams and file formats other than JPEG2000 (O) ..............................................................6 4.9 JPIP request bindings (M).................................6 4.10 Capability exchange and negotiation (O)..................6 4.11 Optimisation of the client server communications (O).....6 4.12 Requesting metadata (O)..................................6 4.13 Uniquely identifying an image (O)........................7 4.14 Uploading images to the server (O).......................7 References.......................................................7 Author's Addresses...............................................7 1. Introduction While JPEG 2000 has many benefits for non-interactive applications, such as image data transmission, the features of JPEG 2000 (e.g. multi-resolution, scalability) are beneficial for interactive applications. A server could deliver a portion of the JPEG 2000 file to satisfy some request from a client. A sensible example would be where the server and client negotiate to transmit only that relevant data such that the client can paint an appropriate rendering of the client's area of interest (AOI) on its display. Thus the server would transmit data for the client's specified area of interest, at a particular resolution and quality level. However, this requires that the client and server are able to have an intelligent "conversation." This document describes the set of requirements for a protocol allowing client-server interaction to control the bandwidth-efficient transmission of JPEG 2000 coded data. The objective of standardizing such a protocol is to promote interoperability between different vendor products. 2. Scope of JPIP This International Standard defines, in an extensible manner, syntaxes and methods for the remote interrogation and optional alteration of JPEG 2000 codestreams and files in accordance with their definition in the following parts of ISO/IEC 15444: - ISO/IEC 15444-1:2000 and its definition of a baseline JPEG 2000 codestream and JP2 file format - The JPEG 2000 family of file formats as defined in further parts of IS 15444 In this document, these are referred to as the JPEG 2000 Interactive Protocol, "JPIP" and interactive applications using JPIP are referred to as a "JPIP system." Prandolini Expires - October 2002 [Page 2] JPIP Requirements and Profiles April 2002 JPIP specifies a protocol where structured series of interactions between a client and a server by which image file metadata, structure and partial or whole image codestreams may be exchanged in a communications efficient manner. These structures include definitions of the semantics and values to be exchanged, and suggest how these may be passed using a variety of existing network transports. With JPIP, the following tasks may be accomplished in varying, compatible ways: - the exchange and negotiation of capability information - the request and transfer of; - selective data segments from a JPEG 2000 file or codestream - selective and defined structures from within a JPEG 2000 file or codestream - parts of an image or its related metadata stored within a JPEG 2000 file or codestream Further, JPIP provides the capability for "fallback," such that the protocol can deliver similar results using differing levels of awareness of JPEG 2000 file and codestream structures at the client and the server. JPIP can be used over a variety of networks and communications media having different characteristics and quality of service characteristics. It can use a number of methods to communicate between client and server, based on existing protocols and network transports, which this International Standard may extend to provide additional JPEG 2000 related functionality. Information which is user or session related may also be exchanged. Its use can be tailored via the various extensions to the JPEG 2000 file format, as defined in ISO/IEC 15444-2, 15444-3 and 15444-6, however these are not mandated to achieve a simple level of interactivity that allows portions of a single JPEG 2000 file or codestream to be transferred. Although the terms "client" and "server" are used in this International Standard to refer to the image receiving and delivering application respectively, it is intended that JPIP can be used within both hierarchical and peer to peer networks, for data delivery in either direction, and for machine to machine as well as user to machine or user to user applications. It is also intended for use as an adjunct to an alternative more comprehensive protocol for image delivery within that protocol, and for the delivery of non-JPEG 2000 coded information. While some features of JPIP may be applied to codestreams which are not JPEG 2000 compliant, such use is not mandated or required by JPIP systems conforming to this International Standard. Prandolini Expires - October 2002 [Page 3] JPIP Requirements and Profiles April 2002 The use of JPIP in an Internet or intranet environment is addressed. Further facilities and features may be available when JPIP is used on top of a basic network transport protocol such as TCP/IP, UDP or RTP. Guidance: There is the possibility that, for some of the processes specified in this International Standard, conformance or compliance may require use of an invention covered by patent rights. So far as WG1 is aware no Intellectual Property claim in connection with JPIP has been made by any of the organizations submitting technology that has been included in this International Standard, and it is the Committee's desire that JPIP should be implementable without payment of license fees or royalties. 3. Markets and applications Which markets are envisioned for this proposed standard? - Digital Photography Databases - Medical Image Databases - Remote Sensing Image Databases - Mobile Imaging - Surveillance 4. Requirements This section contains a list of requirements for JPIP that have been identified by the JPIP system use-case scenarios submitted for target applications. It also contains requirements of a general nature, identified as necessary for successful market adoption. A baseline JPIP system will consist of a selected set of mandatory requirements, identified below. Further, a JPIP system may use additional requirements noted as optional below. The protocol as defined will be extensible to take into account requirements from other parts of the JPEG 2000 standard, specifically JPSEC (Security issues), JP3D (3D image compression and floating point) and JPWL (Wireless issues). At this time, in the definitions below, the label (M) refers to mandatory features that all JPIP systems claiming conformance to JPIP standard must support, and (O) to those optional features which may only be supported by some implementations. The word 'shall' or 'should' within a requirement statement indicates the desirable feature that the JPIP protocol definition should support, even if the requirement is tagged as (O). 4.1 Area of interest selection (M) Prandolini Expires - October 2002 [Page 4] JPIP Requirements and Profiles April 2002 Imaging applications may present a portion of a JPEG 2000 coded image within a "viewport" which has different spatial and component depth resolutions to those of the original image. It should be possible for JPIP to transfer the relevant data necessary to fill the viewport. It should also be possible to transfer metadata and other background information which relates specifically to the area of interest. 4.2 Data transmission efficiency (M) The data transferred by JPIP should be constrained to that requested by the client or mandated by the server (for instance in the case of IPR indications). 4.3 Session persistence (O) It should be possible for JPIP to permit either or both of the client and server to retain information gathered during a series of temporal transactions, even when these may have taken place over a significant period of time, and to use this information to minimise the amount of redundant (i.e. previously transmitted) data exchanged. Methods should be provided within the protocol to identify individual, or a series of, transactions uniquely. 4.4 Error handling (O) It is envisaged that the protocol will be used over a variety of network transports, some of which will have error characteristics that may affect network protocols. JPIP should be able to handle residual errors in a consistent manner with a minimum of error propagation. In particular, in certain applications it may be essential that parts of the data are transferred without error. JPIP should be extensible to cope with error handling requirements of other parts of IS 15444, including JPSEC, JP3D and JPWL. 4.5 Rate-control (O) In certain JPIP systems, such as those where the client has limited processing, addressing or memory resources, it must be possible for the client to constrain the server to only sending data in quantities, or at speeds that it can handle. The protocol should also handle recovery in situations where network caching or delays mean that the client is unable to handle all the data received. 4.6 Support for simultaneous access (O) As JPIP may be used to deliver a mixture of media contents in an effective manner (for instance when used in conjunction with IS 15444-6 JPM files), the protocol should be capable of simultaneous Prandolini Expires - October 2002 [Page 5] JPIP Requirements and Profiles April 2002 use with multiple outstanding requests for image data. It should be possible for the client to synchronise the responses to these requests. 4.7 Retrieve sub-objects from a JPEG 2000 file (M for JPX) The protocol must be aware of the structure of JPEG 2000 files at least to the extent that in can successfully parse and deliver data held within individually identified boxes within each file format as specified by the client. 4.8 Support codestreams and file formats other than JPEG2000 (O) The protocol should also support at least the byte serving of codestreams that are not JPEG2000 codestreams, including those used in IS 15444-6 such as JPEG (IS 10918), T.6, and JBIG2. It may optionally support file format aware communications. 4.9 JPIP request bindings (M) JPIP should allow different request bindings using syntax and primitives defined within the protocol. These might be used to identify objects within either a JPEG 2000 file or codestream, for instance specified by an Uniform Resource Indicator (as defined by RFC2396), or within an HTML encoded page. This is intended to allow the client to make use of IETF and W3C defined Internet standards which are not aware of the structure of JPEG 2000 files or codestreams to transport this information, for example to a browser plug-in. 4.10 Capability exchange and negotiation (O) JPIP shall be able to exchange information about client and/or server capabilities and resources during a session. 4.11 Optimisation of the client server communications (O) JPIP should be able to specify client and/or server resources during a session, permitting renegotiation at any time, and to optimise delivery of information to suit the negotiated session parameters. 4.12 Requesting metadata (O) It shall be possible for JPIP to transfer a catalog of metadata information on request, and for the server to communicate hierarchically ordered metadata. Means should be provided in JPIP for complex metadata queries. Prandolini Expires - October 2002 [Page 6] JPIP Requirements and Profiles April 2002 4.13 Uniquely identifying an image (O) JPIP should be able to use unique identifiers within its protocol stream. These may be linked to images, individual sessions, or other JPEG 2000 objects and structures. Their actual usage in specific instances will be defined within JPIP or other JPEG 2000 standards. 4.14 Uploading images to the server (O) JPIP should offer a method for a client to specify changes to an image held on an server, including creation, amendment and deletion of both file and codestream objects. References - IS 15444. Information Technology. JPEG 2000 Image Coding System Author's Addresses Dr. Robert Prandolini DSTO C3 Research Centre Department of Defence Canberra ACT 2600, Australia Tel: +61 2 6256 6189 Fax: +61 2 6256 6110 E-mail: robert.prandolini@dtso.defence.gov.au Mr. Richard Clark (UK) Elysium Ltd. Milton House Whitehill Road Crowborough East Sussex TN6 1LB, UK Tel: +44 (0) 1982 667411 Fax: +44 (0) 1892 667433 E-mail: richard@elysium.ltd.uk Mr. Scott Houchin (US) Eastman Kodak Company 1700 Dewey Ave Rochester, NY 14650-1919, USA Tel: +1 585 588 8495 Fax: +1 585 588 8509 E-mail: scott.houchin@kodak.com Prandolini Expires - October 2002 [Page 7]