PPSP N. Zong, Ed. Internet-Draft Huawei Technologies Intended status: Standards Track June 28, 2009 Expires: December 30, 2009 Chunk Discovery for P2P Streaming draft-zong-ppsp-chunk-discovery-00 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 30, 2009. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract This document describes several mechanisms of Chunk Discovery in P2P Streaming use case. The Chunk Discovery for P2P streaming provides the functionality of streaming sources registrar and discovery in a Zong Expires December 30, 2009 [Page 1] Internet-Draft P2P Streaming Usage June 2009 fully-distributed system. It provides register, update and lookup service for video chunks in the P2P Streaming applications. The mechanisms include a P2P streaming usage for REsource LOcation And Discovery (RELOAD), and a combined Trakcer and Peer Gossip method. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Implementation using Current RELOAD Mechanisms . . . . . . . . 4 3.1. Register Chunk Information . . . . . . . . . . . . . . . . 5 3.2. PEER-REGISTRATION Kind Definition . . . . . . . . . . . . 6 3.3. Lookup Chunk Information . . . . . . . . . . . . . . . . . 7 3.4. Extend PeerRegistrationData . . . . . . . . . . . . . . . 7 4. Combine Data on the Storage Peer . . . . . . . . . . . . . . . 9 4.1. Register Chunk Information . . . . . . . . . . . . . . . . 9 4.2. Combine Data . . . . . . . . . . . . . . . . . . . . . . . 10 4.3. New PEER-REGISTRATION Kind Definition . . . . . . . . . . 10 4.4. Lookup Chunk Information . . . . . . . . . . . . . . . . . 10 4.5. New Requirement to Storage Peer . . . . . . . . . . . . . 11 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 11 6. A Combined Tracker and Peer Gossip Method . . . . . . . . . . 11 6.1. Tracker Communication . . . . . . . . . . . . . . . . . . 12 6.2. Peer Gossip . . . . . . . . . . . . . . . . . . . . . . . 13 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 8. Security Considerations . . . . . . . . . . . . . . . . . . . 14 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 14 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14 10.1. Normative References . . . . . . . . . . . . . . . . . . . 14 10.2. Informative References . . . . . . . . . . . . . . . . . . 14 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 15 Zong Expires December 30, 2009 [Page 2] Internet-Draft P2P Streaming Usage June 2009 1. Introduction As one of the architectures for deliverying streaming traffic on the global Internet , the P2P streaming has the advantage of being scalable to a large number of subscribers with low infrastructural cost [PPSP1]. P2P streaming applications attract more and more users to consume video content online [PPLive][PPStrm][UUSee]. P2P streaming applications adopt a decentralized streaming architecture where the media content is separated into smaller chunks to be shared among peers. A chunk is a basic unit of partitioned streaming, which is used for the purpose of storage and advertising to neighbors what parts of a movie a peer holds [SIGP2P]. In the case of VoD, different users watch different parts of the video content (i.e. have different chunks) so that a peer normally maintains a buffer to share video chunks with other peers. In the case of live content, because all peers are mostly interested in what is actually happens "now", one possible role of a peer is to request video chunks from a live source and then forward the chunks to more peers. In all the cases, The procedure of chunk discovery for P2P streaming is needed to allow peers to register the information of the video chunks which have been received and ready for sharing with other peers, as well as locate video chunks from multiple peers in a fully-distributed manner. This document describes several mechanisms of Chunk Discovery in P2P Streaming use case. The Chunk Discovery for P2P streaming provides the functionality of streaming sources registrar and discovery in a fully-distributed system. It provides register, update and lookup service for video chunks in the P2P Streaming applications. The mechanisms include a P2P streaming usage for REsource LOcation And Discovery (RELOAD) [I-D.ietf-p2psip-base], and a combined Trakcer and Peer Gossip method [PPSP1][PPSP2]. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. We use the terminology and definitions from Concepts and Terminology for Peer to Peer SIP [I-D.ietf-p2psip-concepts] and the RELOAD Base Protocol [I-D.ietf-p2psip-base]. Zong Expires December 30, 2009 [Page 3] Internet-Draft P2P Streaming Usage June 2009 3. Implementation using Current RELOAD Mechanisms The P2P streaming usage for current RELOAD involves the following functions. 1, Registration. A peer can use the RELOAD data storage functionality to store a mapping from the Chunk-IDs of the stored video chunks to its Node-ID. 2, Lookup. A peer can use the RELOAD data storage functionality to retrieve the Node-IDs of the peers who have registered the information of the requested video chunks. For example, Peer A has received video chunks with Video-ID "Movie A" and Chunk-IDs "Chunk 1", "Chunk 2" and ready for sharing these chunks with other peers. Peer A could register its Node-ID "1234" under these Video-ID and Chunk-IDs. Similarly, Peer B has received video chunks with Video-ID "Movie A" and Chunk-IDs "Chunk 1", "Chunk 2" and "Chunk 3" and register its Node-ID "5678" under these Video-ID and Chunk-IDs. When Peer C wants to match "Movie A" from the chunk with Chunk-ID "Chunk 2", it queries the overlay for "Movie A" and "Chunk 2" and gets back Node-IDs "1234" and "5678". Peer C then uses the overlay to establish a direct connection with Peer A and Peer B and can use that direct connection to perform media signaling process. This example is depicted as follows: Zong Expires December 30, 2009 [Page 4] Internet-Draft P2P Streaming Usage June 2009 Peer A Overlay Peers Peer B Peer C (1234) | (5678) | | | | | |StoreReq(MovieA, | | | | Chunk1/2)-->| | | |<------------StoreAns| | | | |<--StoreReq(MovieA, | | | | Chunk1/2/3)| | | |StoreAns------------>| | | | | | | |<-----------------------FetchReq(MovieA, | | | | Chunk 2)| | | FetchAns(1234/5678)-------------------->| | | | | | | |<----Attach------->| |<----------------------------Attach--------------------------->| | | | | | | |<----ICE Check---->| |<----------------------------ICE Check------------------------>| | | | | | | |<--Media Signal--->| |<------------------------Media Signaling---------------------->| | | | | | | |-----Media-------->| |-----------------------------Media---------------------------->| | | | | 3.1. Register Chunk Information In conventional C/S streaming cases, the media information (e.g. media file URL) can be obtained by media clients from the media server. In P2P streaming usage of RELOAD, each peer registers the information of the video chunks which have been received and ready for sharing with other peers, as well as locate video chunks from multiple peers through the RELOAD overlay. To register its chunk information, a RELOAD peer stores a PeerRegistrationData structure. This uses the PEER-REGISTRATION Kind-ID, which is formally defined in Section 3.2. As a simple example, if Peer A has obtained the video chunks with Video-ID "Movie A" and Chunk-IDs "Chunk 1", "Chunk 2" and its Node-ID is "1234", it may store the mapping "Movie A, Chunk 1/2 -> 1234" in the RELOAD overlay. This would tell anyone who wanted to watch "Movie A" from the Chunk with Chunk-ID "Chunk 1" or "Chunk 2" to contact node "1234". The contents of a PeerRegistrationData structure are as follows: Zong Expires December 30, 2009 [Page 5] Internet-Draft P2P Streaming Usage June 2009 struct { uint16 destination_id_list_length; Destination destination_id_list[destination_id_list_length]; } PeerRegistrationData; The contents of the PeerRegistrationData PDU are: destination_id_list_length: The number of the peers having registered the chunk. destination_id_list: The identifiers of peers who have registered the chunk. Note that RELOAD explicitly supports that multiple peers registration the same combination of Video-ID and Chunk-ID. For example, both Peer A and Peer B could register "Movie A" and "Chunk 1". Therefore, the registrations are stored in a Dictionary with dictionary values being PeerRegistrationData and dictionary keys being Node-IDs. So, let's say Peer A (with Node-ID "1234") has "Chunk 1" and "Chunk 2" of "Movie A" and Peer B (with Node-ID "5678") has "Chunk 1", "Chunk 2" and "Chunk 3" of "Movie A". Peer A would generate PeerResgistrationData of {1, [1234]}, and PeerB would create PeerRegistrationData {1, [5678]}. Then they would do the following stores (registers) into the overlay: Peer A: Movie A:Chunk 1 -> 1234, {1, [1234] } Movie A:Chunk 2 -> 1234, {1, [1234] } Peer B: Movie A:Chunk 1 -> 5678, {1, [5678] } Movie A:Chunk 2 -> 5678, {1, [5678] } Movie A:Chunk 3 -> 5678, {1, [5678] } 3.2. PEER-REGISTRATION Kind Definition PEER-REGISTRATION Kind-ID include: 1, Kind IDs. The Resource Name for the PEER-REGISTRATION is the combination of Video-ID and Chunk-ID. The data stored is a PeerRegistrationData, which contains the Node-IDs of peers who can provide the video chunks. Zong Expires December 30, 2009 [Page 6] Internet-Draft P2P Streaming Usage June 2009 2, Data Model. The data model for the PEER-REGISTRATION Kind is dictionary. The dictionary key is the Node-ID of the peer who can provide the video chunk. 3, Access Control. If certificate-based access control is being used, stored data of Kind PEER-REGISTRATION must be signed by a certificate which contains a Node-ID matching the storing dictionary key. 3.3. Lookup Chunk Information In P2P streaming usage of RELOAD, when a peer wants to match a video from certain chunk, it queries the overlay for the corresponding Video-ID and Chunk-ID and gets back the Node-IDs of the peers who can provide the video chunks. As a simple example, if Peer C wants to match "Movie A" from the chunk with Chunk-ID "Chunk 2", it performs a Fetch for kind PEER- REGISTRATION at Video-ID "Movie A" and Chunk-ID "Chunk 2". If this Fetch does not indicate any specific dictionary keys, it will result in fetching all the stored values, which is PeerRegistrationData as follows: Dictionary Results: 1234, {1, [1234] } 5678, {1, [5678] } Peer C then uses the overlay to establish direct connections with Peer A and Peer B and can use these direct connections to perform media signaling process and video transportation. 3.4. Extend PeerRegistrationData Note that the performance of chunk discovery would be increased by adding some fields to the PeerRegostrationData structure that lets a peer list all the chunks it has. That is, if Peer C searches for "Movie A" and "Chunk 2", it will find not only that both Peer A and Peer B have "Movie A" and "Chunk 2", but aslo Peer A has "Chunk 1" and Peer B has "Chunk 1", "Chunk 3". Peer C then would probably directly request Peer B to serve "Chunk 3" after "Chunk 2", without asking for the RELOAD to do a new lookup for "Chunk 3". The contents of the extended PeerRegistrationData structure are as follows: Zong Expires December 30, 2009 [Page 7] Internet-Draft P2P Streaming Usage June 2009 typedef opaque ChunkID[2]; struct { Destination destination_id; uint16 chunk_id_list_length; ChunkID chunk_id_list[chunk_id_list_length]; } DestChunkData struct { uint16 destination_id_list_length; DestChunkData destination_chunk_list[destination_id_list_length]; } PeerRegistrationData; The contents of the DestChunkData PDU are: ChunkID: A fixed-length 16-bit string. chunk_id_list_length: The number of the chunks owned by the peer. chunk_id_list: The identifiers of chunks owned by the peer. The contents of the extened PeerRegistrationData PDU are: destination_id_list_length: The number of the peers having registered the chunk. destination_chunk_list: The list of DestChunkData corresponding to the peers that register the chunk. Therefore, in the previous example, Peer A and Peer B would do the following stores (registers) into the overlay: Peer A: Movie A:Chunk 1 -> 1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] } Movie A:Chunk 2 -> 1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] } Peer B: Movie A:Chunk 1 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Movie A:Chunk 2 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Movie A:Chunk 3 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Zong Expires December 30, 2009 [Page 8] Internet-Draft P2P Streaming Usage June 2009 Similarly, if Peer C wants to match "Movie A" and "Chunk 2", it gets back the following results: Dictionary Results: 1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] } 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } 4. Combine Data on the Storage Peer However, if there are many chunks (say more than 1000) in a movie, then a peer with the whole movie will have to perform many registrations to list all the things that it has registered. This would impact the RELOAD overlay as an inefficient way to do the chunk registration. 4.1. Register Chunk Information Now we define a new ChunkInfoData structure as follows: typedef opaque ChunkID[2]; struct { uint16 chunk_id_list_length; ChunkID chunk_id_list[chunk_id_list_length]; } ChunkInfoData; The contents of the ChunkInfoData PDU are: ChunkID: A fixed-length 16-bit string. chunk_id_list_length: The number of the chunks owned by the peer. chunk_id_list: The identifiers of chunks owned by the peer. Let's reuse the example in Section 3.1. Suppose that Peer A and Peer B both do the following stores (registers) into the overlay: Peer A: Movie A -> 1234, {2, [Chunk1,Chunk2]} Peer B: Movie A -> 5678, {3, [Chunk1,Chunk2,Chunk3]} Zong Expires December 30, 2009 [Page 9] Internet-Draft P2P Streaming Usage June 2009 It can be observed that the number of registrations performed by peers is largely reduced. 4.2. Combine Data The storage peer needs to re-arrage the stored data to: Movie A -> Chunk1, {2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Movie A -> Chunk2, {2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Movie A -> Chunk3, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } So, the registrations are stored in a Dictionary with dictionary values being PeerRegistrationData and dictionary keys being Chunk- IDs. 4.3. New PEER-REGISTRATION Kind Definition The new PEER-REGISTRATION Kind-ID include: 1, Kind IDs. The Resource Name for the new PEER-REGISTRATION is the Video-ID. The data stored is a PeerRegistrationData, which contains the Node-IDs of the peers who can provide the video. 2, Data Model. The data model for the new PEER-REGISTRATION Kind is dictionary. The dictionary key is the Chunk-ID of the chunk registered by the peers who can provide the video chunk. 3, Access Control. If certificate-based access control is being used, stored data of the new Kind PEER-REGISTRATION must be signed by a certificate which contains a Chunk-ID matching the storing dictionary key. 4.4. Lookup Chunk Information If Peer C wants to match "Movie A" from the chunk with Chunk-ID "Chunk 2", it performs a Fetch for kind PEER-REGISTRATION at Video-ID "Movie A" with dictionary key "Chunk2". It will send back dictionary results as follows: Dictionary Results: {2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] } Zong Expires December 30, 2009 [Page 10] Internet-Draft P2P Streaming Usage June 2009 Peer C then uses the overlay to establish direct connections with Peer A and Peer B and can use these direct connections to perform media signaling process and video transportation. 4.5. New Requirement to Storage Peer It is now obviously required that the storage peer performes the combining of the chunk information data (with the data structure of ChunkInfoData) from different peers and convert them to the data structure of PeerRegistrationData. This capability of combining and converting data between different data structures would be a new functionality of RELOAD. 5. Open Issues Although the P2P streaming usage of RELOAD would be straightforward, it is still an open question that if the RELOAD is suitable for P2P streaming. 1, Searching Efficiency. P2P streaming applications normally require retrieving the real-time/para real-time video chunks quickly through the direct chunk information exchange among the peers. On the other hand, iterative and recursive routing process supported in RELAOD may not meet the real-time requirement of the P2P streaming [PPSP2]. 2, Working Load. P2P streaming applications need constant and more frequent support of RELOAD. On one hand, during the P2P streaming process, the requestor would constantly ask for RELOAD to lookup a new set of peers to serve next chunks of video. This is either because the current peers left or the requestor is not sure if the current peers can serve the next chunks of video. On the other hand, peers also need to constantly advise the chunks they have to the RELOAD to keep the chunk information updated on the RELOAD overlay. The need of constant support of RELOAD may impose a large burden on RELOAD, supposing that there are a large number (say hundreds of thousands) of simultaneously on-line P2P streaming peers. These issue of the performance of RELOAD on supporting P2P streaming applications need to be evaluated in both PPSP and P2PSIP communities. 6. A Combined Tracker and Peer Gossip Method The combined tracker and peer gossip method generally contains Zong Expires December 30, 2009 [Page 11] Internet-Draft P2P Streaming Usage June 2009 following components: 1. tracker communications. Peer reports to some centralized directory server (a.k.a. tracker) coarse information (i.e. Video-ID). When the tracker receives the video request from a peer, it returns to the requesting peer with a peer list (or, a swarm) in which the peers are sharing the requested video. 2. peer gossip. The grain information (i.e. Chunk-ID) is achieved by the requesting peer through directly communicating with the peers in the obtained peer list to exchange the chunk description. +---------------+ ------------>| Tracker |<------------------------ | +---------------+ | | | | | | Tracker Communications | | | | | V V | +--------+ Peer Gossip (Chunk Description) +--------+ | | |<------------------------------->| | | | Peer | | Peer | | | |<--------------------- | | | +--------+ | +--------+ | Peer Gossip | V (Chunk Description) | +--------+ | | | ---------------------->| Peer | | | +--------+ More Peers ... ... 6.1. Tracker Communication For example, Peer A is watching a video "Movie A" and ready for sharing this video with other peers. Peer A could report to the tracker by sending the Video-ID "Movie A" to the tracker. Similarly, Peer B is watching "Movie A" and register itself under the Video-ID "Movie A" to the tracker. Tracker then keep a record that both Peer A and Peer B would share "Movie A" with other peers. When Peer C wants to match "Movie A" , it firstly queries the tracker for the list of peers who would provide the video. It gets back Peer A and Peer B. Peer C then would communicate with both Peer A and Peer B to check who would serve the requested chunks of "Movie A". This exaple Zong Expires December 30, 2009 [Page 12] Internet-Draft P2P Streaming Usage June 2009 is shown as below: Peer A racker Peer B Peer C | | | | |Report(Movie A)--->| | | |<---------ReportAns| | | | | | | | |<---Report(Movie A)| | | |ReportAns--------->| | | | | | | |<-----------------------Request(Movie A)| | |RequestAns(Peer A/Peer B)-------------->| | | | | 6.2. Peer Gossip For example, even Peer C gets back the peer list including Peer A and Peer B, it still doesn't know who can serve the requested chunk (say "Chunk 3"). Therefore, Peer C needs to discover which peer would serve the request chunk of "Movie A". The chunk information (i.e. whick chunks are stored on which peers) can be achieved by directly communicating with both Peer A and Peer B to exchange the chunk description. Suppose that Peer A stores video chunks with Chunk-ID "Chunk 1", "Chunk 2" and Peer B stored "Chunk 1", "Chunk 2" and "Chunk 3". This exaple is illustrated as follows: Peer A Peer B Peer C | | | | <------------------------------------------ChunkDescReq | | ChunkDescAns(Chunk 1/2)-------------------------------> | | | | | | <--------------ChunkDescReq | | | ChunkDescAns(Chunk 1/2/3)-> | | | | | | <-----Media Signaling-----> | | | | | | ------Media---------------> | | | | Because only Peer B has the chunk with Chunk-ID of "Chunk 3", Peer C then requests the data of "Chunk 3" from Peer B by establishing a media connection with Peer B. 7. IANA Considerations TODO define PEER-REGISTRATION Kind-ID in the case of using RELOAD for P2P streaming chunk discovery, Zong Expires December 30, 2009 [Page 13] Internet-Draft P2P Streaming Usage June 2009 8. Security Considerations TODO describe security considerations when performing chunk discovery for P2P streaming. 9. Acknowledgements The authors would like to thank the following for their contributions to this document: David Bryan. TODO list more. 10. References 10.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [I-D.ietf-p2psip-base] Jennings, C., Lowekamp, B., Rescorla, E., Baset, S., and H. Schulzrinne, "REsource LOcation And Discovery (RELOAD) Base Protocol", draft-ietf-p2psip-base-02 (work in progress), March 2009. [I-D.ietf-p2psip-concepts] Bryan, D., Matthews, P., Shim, E., Willis, D., and S. Dawkins, "Concepts and Terminology for Peer to Peer SIP", draft-ietf-p2psip-concepts-02 (work in progress), July 2008. 10.2. Informative References [PPLive] "www.pplive.com". [PPStrm] "www.ppstream.com". [UUSee] "www.uusee.com". [SIGP2P] Huang, "Challenges, Design and Analysis of a Large-scale P2P-VoD System". [PPSP1] Zong, N. and X. Jiang, "Survey and Problem Statement of P2P Streaming", IETF PPSP BoF, November 2008. [PPSP2] Zhang, Y., Zong, N., Camarillo, G., Seng, J., and R. Yang, "Problem Statement of P2P Streaming Protocol (PPSP)", IETF PPSP BoF, May 2009. Zong Expires December 30, 2009 [Page 14] Internet-Draft P2P Streaming Usage June 2009 Author's Address Ning Zong (editor) Huawei Technologies 10F, Huihong Mansion, No.91 Baixia Road Nanjing 210001 China Phone: +86 25 84565866 Email: zongning@huawei.com Zong Expires December 30, 2009 [Page 15]