SPEECHSC S. Shanmugham Internet-Draft Cisco Systems, Inc. Expires: December 11, 2006 D. Burnett Vocalocity Inc. June 9, 2006 Media Resource Control Protocol Version 2 (MRCPv2) draft-ietf-speechsc-mrcpv2-10 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 11, 2006. Copyright Notice Copyright (C) The Internet Society (2006). Abstract The MRCPv2 protocol allows client hosts to control media service resources such as speech synthesizers, recognizers, verifiers and identifiers residing in servers on the network. MRCPv2 is not a "stand-alone" protocol - it relies on a session management protocol such as the Session Initiation Protocol (SIP) to establish the MRCPv2 control session between the client and the server, and for rendezvous and capability discovery. It also depends on SIP and SDP to Shanmugham & Burnett Expires December 11, 2006 [Page 1] Internet-Draft MRCPv2 June 2006 establish the media sessions and associated parameters between the media source or sink and the media server. Once this is done, the MRCPv2 protocol exchange operates over the control session established above, allowing the client to control the media processing resources on the speech resource server. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 8 2. Document Conventions . . . . . . . . . . . . . . . . . . . . 9 2.1. Definitions . . . . . . . . . . . . . . . . . . . . . . 9 3. Architecture . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1. MRCPv2 Media Resource Types . . . . . . . . . . . . . . 11 3.2. Server and Resource Addressing . . . . . . . . . . . . . 12 4. MRCPv2 Protocol Basics . . . . . . . . . . . . . . . . . . . 12 4.1. Connecting to the Server . . . . . . . . . . . . . . . . 13 4.2. Managing Resource Control Channels . . . . . . . . . . . 13 4.3. Media Streams and RTP Ports . . . . . . . . . . . . . . 19 4.4. MRCPv2 Message Transport . . . . . . . . . . . . . . . . 21 5. MRCPv2 Specification . . . . . . . . . . . . . . . . . . . . 21 5.1. Common Protocol Elements . . . . . . . . . . . . . . . . 22 5.2. Request . . . . . . . . . . . . . . . . . . . . . . . . 23 5.3. Response . . . . . . . . . . . . . . . . . . . . . . . . 24 5.4. Status Codes . . . . . . . . . . . . . . . . . . . . . . 25 5.5. Events . . . . . . . . . . . . . . . . . . . . . . . . . 26 6. MRCPv2 Generic Methods and Headers . . . . . . . . . . . . . 27 6.1. Generic Methods . . . . . . . . . . . . . . . . . . . . 27 6.1.1. SET-PARAMS . . . . . . . . . . . . . . . . . . . . . 27 6.1.2. GET-PARAMS . . . . . . . . . . . . . . . . . . . . . 28 6.2. Generic Message Headers . . . . . . . . . . . . . . . . 29 6.2.1. Channel-Identifier . . . . . . . . . . . . . . . . . 30 6.2.2. Accept . . . . . . . . . . . . . . . . . . . . . . . 30 6.2.3. Active-Request-Id-List . . . . . . . . . . . . . . . 31 6.2.4. Proxy-Sync-Id . . . . . . . . . . . . . . . . . . . 31 6.2.5. Accept-Charset . . . . . . . . . . . . . . . . . . . 32 6.2.6. Content-Type . . . . . . . . . . . . . . . . . . . . 32 6.2.7. Content-ID . . . . . . . . . . . . . . . . . . . . . 32 6.2.8. Content-Base . . . . . . . . . . . . . . . . . . . . 32 6.2.9. Content-Encoding . . . . . . . . . . . . . . . . . . 32 6.2.10. Content-Location . . . . . . . . . . . . . . . . . . 33 6.2.11. Content-Length . . . . . . . . . . . . . . . . . . . 33 6.2.12. Fetch Timeout . . . . . . . . . . . . . . . . . . . 34 6.2.13. Cache-Control . . . . . . . . . . . . . . . . . . . 34 6.2.14. Logging-Tag . . . . . . . . . . . . . . . . . . . . 35 6.2.15. Set-Cookie and Set-Cookie2 . . . . . . . . . . . . . 36 6.2.16. Vendor Specific Parameters . . . . . . . . . . . . . 37 7. Resource Discovery . . . . . . . . . . . . . . . . . . . . . 38 Shanmugham & Burnett Expires December 11, 2006 [Page 2] Internet-Draft MRCPv2 June 2006 8. Speech Synthesizer Resource . . . . . . . . . . . . . . . . . 39 8.1. Synthesizer State Machine . . . . . . . . . . . . . . . 40 8.2. Synthesizer Methods . . . . . . . . . . . . . . . . . . 41 8.3. Synthesizer Events . . . . . . . . . . . . . . . . . . . 41 8.4. Synthesizer Header Fields . . . . . . . . . . . . . . . 42 8.4.1. Jump-Size . . . . . . . . . . . . . . . . . . . . . 42 8.4.2. Kill-On-Barge-In . . . . . . . . . . . . . . . . . . 43 8.4.3. Speaker Profile . . . . . . . . . . . . . . . . . . 43 8.4.4. Completion Cause . . . . . . . . . . . . . . . . . . 44 8.4.5. Completion Reason . . . . . . . . . . . . . . . . . 44 8.4.6. Voice-Parameters . . . . . . . . . . . . . . . . . . 45 8.4.7. Prosody-Parameters . . . . . . . . . . . . . . . . . 45 8.4.8. Speech Marker . . . . . . . . . . . . . . . . . . . 46 8.4.9. Speech Language . . . . . . . . . . . . . . . . . . 46 8.4.10. Fetch Hint . . . . . . . . . . . . . . . . . . . . . 46 8.4.11. Audio Fetch Hint . . . . . . . . . . . . . . . . . . 47 8.4.12. Failed URI . . . . . . . . . . . . . . . . . . . . . 47 8.4.13. Failed URI Cause . . . . . . . . . . . . . . . . . . 47 8.4.14. Speak Restart . . . . . . . . . . . . . . . . . . . 47 8.4.15. Speak Length . . . . . . . . . . . . . . . . . . . . 48 8.4.16. Load-Lexicon . . . . . . . . . . . . . . . . . . . . 48 8.4.17. Lexicon-Search-Order . . . . . . . . . . . . . . . . 48 8.5. Synthesizer Message Body . . . . . . . . . . . . . . . . 49 8.5.1. Synthesizer Speech Data . . . . . . . . . . . . . . 49 8.5.2. Lexicon Data . . . . . . . . . . . . . . . . . . . . 51 8.6. SPEAK Method . . . . . . . . . . . . . . . . . . . . . . 52 8.7. STOP . . . . . . . . . . . . . . . . . . . . . . . . . . 53 8.8. BARGE-IN-OCCURED . . . . . . . . . . . . . . . . . . . . 55 8.9. PAUSE . . . . . . . . . . . . . . . . . . . . . . . . . 57 8.10. RESUME . . . . . . . . . . . . . . . . . . . . . . . . . 58 8.11. CONTROL . . . . . . . . . . . . . . . . . . . . . . . . 59 8.12. SPEAK-COMPLETE . . . . . . . . . . . . . . . . . . . . . 62 8.13. SPEECH-MARKER . . . . . . . . . . . . . . . . . . . . . 62 8.14. DEFINE-LEXICON . . . . . . . . . . . . . . . . . . . . . 64 9. Speech Recognizer Resource . . . . . . . . . . . . . . . . . 65 9.1. Recognizer State Machine . . . . . . . . . . . . . . . . 66 9.2. Recognizer Methods . . . . . . . . . . . . . . . . . . . 67 9.3. Recognizer Events . . . . . . . . . . . . . . . . . . . 68 9.4. Recognizer Header Fields . . . . . . . . . . . . . . . . 68 9.4.1. Confidence Threshold . . . . . . . . . . . . . . . . 70 9.4.2. Sensitivity Level . . . . . . . . . . . . . . . . . 70 9.4.3. Speed Vs Accuracy . . . . . . . . . . . . . . . . . 71 9.4.4. N Best List Length . . . . . . . . . . . . . . . . . 71 9.4.5. Input Type . . . . . . . . . . . . . . . . . . . . . 71 9.4.6. No Input Timeout . . . . . . . . . . . . . . . . . . 71 9.4.7. Recognition Timeout . . . . . . . . . . . . . . . . 72 9.4.8. Waveform URI . . . . . . . . . . . . . . . . . . . . 72 9.4.9. Media Type . . . . . . . . . . . . . . . . . . . . . 73 Shanmugham & Burnett Expires December 11, 2006 [Page 3] Internet-Draft MRCPv2 June 2006 9.4.10. Input-Waveform-URI . . . . . . . . . . . . . . . . . 73 9.4.11. Completion Cause . . . . . . . . . . . . . . . . . . 73 9.4.12. Completion Reason . . . . . . . . . . . . . . . . . 75 9.4.13. Recognizer Context Block . . . . . . . . . . . . . . 76 9.4.14. Start Input Timers . . . . . . . . . . . . . . . . . 76 9.4.15. Speech Complete Timeout . . . . . . . . . . . . . . 77 9.4.16. Speech Incomplete Timeout . . . . . . . . . . . . . 77 9.4.17. DTMF Interdigit Timeout . . . . . . . . . . . . . . 78 9.4.18. DTMF Term Timeout . . . . . . . . . . . . . . . . . 78 9.4.19. DTMF-Term-Char . . . . . . . . . . . . . . . . . . . 78 9.4.20. Failed URI . . . . . . . . . . . . . . . . . . . . . 78 9.4.21. Failed URI Cause . . . . . . . . . . . . . . . . . . 78 9.4.22. Save Waveform . . . . . . . . . . . . . . . . . . . 79 9.4.23. New Audio Channel . . . . . . . . . . . . . . . . . 79 9.4.24. Speech-Language . . . . . . . . . . . . . . . . . . 79 9.4.25. Ver-Buffer-Utterance . . . . . . . . . . . . . . . . 80 9.4.26. Recognition-Mode . . . . . . . . . . . . . . . . . . 80 9.4.27. Cancel-If-Queue . . . . . . . . . . . . . . . . . . 80 9.4.28. Hotword-Max-Duration . . . . . . . . . . . . . . . . 81 9.4.29. Hotword-Min-Duration . . . . . . . . . . . . . . . . 81 9.4.30. Interpret-Text . . . . . . . . . . . . . . . . . . . 81 9.4.31. DTMF-Buffer-Time . . . . . . . . . . . . . . . . . . 81 9.4.32. Clear-DTMF-Buffer . . . . . . . . . . . . . . . . . 82 9.4.33. Early-No-Match . . . . . . . . . . . . . . . . . . . 82 9.4.34. Num-Min-Consistent-Pronunciations . . . . . . . . . 82 9.4.35. Consistency-Threshold . . . . . . . . . . . . . . . 82 9.4.36. Clash-Threshold . . . . . . . . . . . . . . . . . . 83 9.4.37. Personal-Grammar-URI . . . . . . . . . . . . . . . . 83 9.4.38. Enroll-Utterance . . . . . . . . . . . . . . . . . . 83 9.4.39. Phrase-Id . . . . . . . . . . . . . . . . . . . . . 84 9.4.40. Phrase-NL . . . . . . . . . . . . . . . . . . . . . 84 9.4.41. Weight . . . . . . . . . . . . . . . . . . . . . . . 84 9.4.42. Save-Best-Waveform . . . . . . . . . . . . . . . . . 84 9.4.43. New-Phrase-Id . . . . . . . . . . . . . . . . . . . 85 9.4.44. Confusable-Phrases-URI . . . . . . . . . . . . . . . 85 9.4.45. Abort-Phrase-Enrollment . . . . . . . . . . . . . . 85 9.5. Recognizer Message Body . . . . . . . . . . . . . . . . 85 9.5.1. Recognizer Grammar Data . . . . . . . . . . . . . . 86 9.5.2. Recognizer Result Data . . . . . . . . . . . . . . . 89 9.5.3. Enrollment Result Data . . . . . . . . . . . . . . . 90 9.5.4. Recognizer Context Block . . . . . . . . . . . . . . 90 9.6. Natural Language Semantic Markup Language . . . . . . . 90 9.6.1. Markup Functions . . . . . . . . . . . . . . . . . . 91 9.6.2. Overview of NLSML Elements and their Relationships . 92 9.6.3. Elements and Attributes . . . . . . . . . . . . . . 92 9.7. Enrollment Results . . . . . . . . . . . . . . . . . . . 97 9.7.1. NUM-CLASHES Element . . . . . . . . . . . . . . . . 98 9.7.2. NUM-GOOD-REPETITIONS Element . . . . . . . . . . . . 98 Shanmugham & Burnett Expires December 11, 2006 [Page 4] Internet-Draft MRCPv2 June 2006 9.7.3. NUM-REPETITIONS-STILL-NEEDED Element . . . . . . . . 98 9.7.4. CONSISTENCY-STATUS Element . . . . . . . . . . . . . 98 9.7.5. CLASH-PHRASE-IDS Element . . . . . . . . . . . . . . 98 9.7.6. TRANSCRIPTIONS Element . . . . . . . . . . . . . . . 98 9.7.7. CONFUSABLE-PHRASES Element . . . . . . . . . . . . . 99 9.8. DEFINE-GRAMMAR . . . . . . . . . . . . . . . . . . . . . 99 9.9. RECOGNIZE . . . . . . . . . . . . . . . . . . . . . . . 103 9.10. STOP . . . . . . . . . . . . . . . . . . . . . . . . . . 108 9.11. GET-RESULT . . . . . . . . . . . . . . . . . . . . . . . 109 9.12. START-OF-INPUT . . . . . . . . . . . . . . . . . . . . . 110 9.13. START-INPUT-TIMERS . . . . . . . . . . . . . . . . . . . 111 9.14. RECOGNITION-COMPLETE . . . . . . . . . . . . . . . . . . 111 9.15. START-PHRASE-ENROLLMENT . . . . . . . . . . . . . . . . 113 9.16. ENROLLMENT-ROLLBACK . . . . . . . . . . . . . . . . . . 114 9.17. END-PHRASE-ENROLLMENT . . . . . . . . . . . . . . . . . 115 9.18. MODIFY-PHRASE . . . . . . . . . . . . . . . . . . . . . 115 9.19. DELETE-PHRASE . . . . . . . . . . . . . . . . . . . . . 116 9.20. INTERPRET . . . . . . . . . . . . . . . . . . . . . . . 116 9.21. INTERPRETATION-COMPLETE . . . . . . . . . . . . . . . . 117 9.22. DTMF Detection . . . . . . . . . . . . . . . . . . . . . 119 10. Recorder Resource . . . . . . . . . . . . . . . . . . . . . . 119 10.1. Recorder State Machine . . . . . . . . . . . . . . . . . 119 10.2. Recorder Methods . . . . . . . . . . . . . . . . . . . . 120 10.3. Recorder Events . . . . . . . . . . . . . . . . . . . . 120 10.4. Recorder Header Fields . . . . . . . . . . . . . . . . . 120 10.4.1. Sensitivity Level . . . . . . . . . . . . . . . . . 120 10.4.2. No Input Timeout . . . . . . . . . . . . . . . . . . 121 10.4.3. Completion Cause . . . . . . . . . . . . . . . . . . 121 10.4.4. Completion Reason . . . . . . . . . . . . . . . . . 121 10.4.5. Failed URI . . . . . . . . . . . . . . . . . . . . . 122 10.4.6. Failed URI Cause . . . . . . . . . . . . . . . . . . 122 10.4.7. Record URI . . . . . . . . . . . . . . . . . . . . . 122 10.4.8. Media Type . . . . . . . . . . . . . . . . . . . . . 123 10.4.9. Max Time . . . . . . . . . . . . . . . . . . . . . . 123 10.4.10. Trim-Length . . . . . . . . . . . . . . . . . . . . 123 10.4.11. Final Silence . . . . . . . . . . . . . . . . . . . 123 10.4.12. Capture On Speech . . . . . . . . . . . . . . . . . 124 10.4.13. Ver-Buffer-Utterance . . . . . . . . . . . . . . . . 124 10.4.14. Start Input Timers . . . . . . . . . . . . . . . . . 124 10.4.15. New Audio Channel . . . . . . . . . . . . . . . . . 124 10.5. Recorder Message Body . . . . . . . . . . . . . . . . . 124 10.6. RECORD . . . . . . . . . . . . . . . . . . . . . . . . . 125 10.7. STOP . . . . . . . . . . . . . . . . . . . . . . . . . . 126 10.8. RECORD-COMPLETE . . . . . . . . . . . . . . . . . . . . 127 10.9. START-INPUT-TIMERS . . . . . . . . . . . . . . . . . . . 128 10.10. START-OF-INPUT . . . . . . . . . . . . . . . . . . . . . 128 11. Speaker Verification and Identification . . . . . . . . . . . 128 11.1. Speaker Verification State Machine . . . . . . . . . . . 130 Shanmugham & Burnett Expires December 11, 2006 [Page 5] Internet-Draft MRCPv2 June 2006 11.2. Speaker Verification Methods . . . . . . . . . . . . . . 131 11.3. Verification Events . . . . . . . . . . . . . . . . . . 132 11.4. Verification Header Fields . . . . . . . . . . . . . . . 133 11.4.1. Repository-URI . . . . . . . . . . . . . . . . . . . 133 11.4.2. Voiceprint-Identifier . . . . . . . . . . . . . . . 133 11.4.3. Verification-Mode . . . . . . . . . . . . . . . . . 134 11.4.4. Adapt-Model . . . . . . . . . . . . . . . . . . . . 135 11.4.5. Abort-Model . . . . . . . . . . . . . . . . . . . . 135 11.4.6. Min-Verification-Score . . . . . . . . . . . . . . . 135 11.4.7. Num-Min-Verification-Phrases . . . . . . . . . . . . 136 11.4.8. Num-Max-Verification-Phrases . . . . . . . . . . . . 136 11.4.9. No-Input-Timeout . . . . . . . . . . . . . . . . . . 136 11.4.10. Save-Waveform . . . . . . . . . . . . . . . . . . . 136 11.4.11. Media Type . . . . . . . . . . . . . . . . . . . . . 137 11.4.12. Waveform-URI . . . . . . . . . . . . . . . . . . . . 137 11.4.13. Voiceprint-Exists . . . . . . . . . . . . . . . . . 137 11.4.14. Ver-Buffer-Utterance . . . . . . . . . . . . . . . . 138 11.4.15. Input-Waveform-Uri . . . . . . . . . . . . . . . . . 138 11.4.16. Completion-Cause . . . . . . . . . . . . . . . . . . 138 11.4.17. Completion Reason . . . . . . . . . . . . . . . . . 140 11.4.18. Speech Complete Timeout . . . . . . . . . . . . . . 140 11.4.19. New Audio Channel . . . . . . . . . . . . . . . . . 140 11.4.20. Abort-Verification . . . . . . . . . . . . . . . . . 140 11.4.21. Start Input Timers . . . . . . . . . . . . . . . . . 140 11.5. Verification Result Elements . . . . . . . . . . . . . . 141 11.5.1. Voiceprint . . . . . . . . . . . . . . . . . . . . . 143 11.5.2. Cumulative . . . . . . . . . . . . . . . . . . . . . 144 11.5.3. Incremental . . . . . . . . . . . . . . . . . . . . 144 11.5.4. Decision . . . . . . . . . . . . . . . . . . . . . . 144 11.5.5. Utterance-Length . . . . . . . . . . . . . . . . . . 144 11.5.6. Device . . . . . . . . . . . . . . . . . . . . . . . 144 11.5.7. Gender . . . . . . . . . . . . . . . . . . . . . . . 144 11.5.8. Adapted . . . . . . . . . . . . . . . . . . . . . . 144 11.5.9. Verification-Score . . . . . . . . . . . . . . . . . 145 11.5.10. Vendor-Specific-Results . . . . . . . . . . . . . . 145 11.6. START-SESSION . . . . . . . . . . . . . . . . . . . . . 145 11.7. END-SESSION . . . . . . . . . . . . . . . . . . . . . . 146 11.8. QUERY-VOICEPRINT . . . . . . . . . . . . . . . . . . . . 147 11.9. DELETE-VOICEPRINT . . . . . . . . . . . . . . . . . . . 148 11.10. VERIFY . . . . . . . . . . . . . . . . . . . . . . . . . 149 11.11. VERIFY-FROM-BUFFER . . . . . . . . . . . . . . . . . . . 149 11.12. VERIFY-ROLLBACK . . . . . . . . . . . . . . . . . . . . 152 11.13. STOP . . . . . . . . . . . . . . . . . . . . . . . . . . 152 11.14. START-INPUT-TIMERS . . . . . . . . . . . . . . . . . . . 153 11.15. VERIFICATION-COMPLETE . . . . . . . . . . . . . . . . . 153 11.16. START-OF-INPUT . . . . . . . . . . . . . . . . . . . . . 154 11.17. CLEAR-BUFFER . . . . . . . . . . . . . . . . . . . . . . 154 11.18. GET-INTERMEDIATE-RESULT . . . . . . . . . . . . . . . . 155 Shanmugham & Burnett Expires December 11, 2006 [Page 6] Internet-Draft MRCPv2 June 2006 12. Security Considerations . . . . . . . . . . . . . . . . . . . 156 12.1. Rendezvous and Session Establishment . . . . . . . . . . 157 12.2. Control channel protection . . . . . . . . . . . . . . . 157 12.3. Media session protection . . . . . . . . . . . . . . . . 157 12.4. Indirect Content Access . . . . . . . . . . . . . . . . 157 12.5. Protection of stored media . . . . . . . . . . . . . . . 158 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 158 13.1. New registries . . . . . . . . . . . . . . . . . . . . . 158 13.1.1. MRCPv2 resource types . . . . . . . . . . . . . . . 158 13.1.2. MRCPv2 methods and events . . . . . . . . . . . . . 158 13.1.3. MRCPv2 headers . . . . . . . . . . . . . . . . . . . 158 13.1.4. MRCPv2 status codes . . . . . . . . . . . . . . . . 159 13.1.5. Grammar Reference List Parameters . . . . . . . . . 159 13.1.6. MRCPv2 vendor-specific parameters . . . . . . . . . 159 13.2. NLSML-related registrations . . . . . . . . . . . . . . 159 13.2.1. application/nlsml+xml MIME type registration . . . . 160 13.3. NLSML XML DTD registration . . . . . . . . . . . . . . . 160 13.4. NLSML XML Schema registration . . . . . . . . . . . . . 161 13.5. NLSML XML Name space registration . . . . . . . . . . . 161 13.6. text/grammar-ref-list Mime Type Registration . . . . . . 161 13.7. session URL scheme registration . . . . . . . . . . . . 162 13.8. SDP parameter registrations . . . . . . . . . . . . . . 163 14. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 164 14.1. Message Flow . . . . . . . . . . . . . . . . . . . . . . 164 14.2. Recognition Result Examples . . . . . . . . . . . . . . 173 14.2.1. Simple ASR Ambiguity . . . . . . . . . . . . . . . . 173 14.2.2. Mixed Initiative . . . . . . . . . . . . . . . . . . 173 14.2.3. DTMF Input . . . . . . . . . . . . . . . . . . . . . 174 14.2.4. Interpreting Meta-Dialog and Meta-Task Utterances . 175 14.2.5. Anaphora and Deixis . . . . . . . . . . . . . . . . 176 14.2.6. Distinguishing Individual Items from Sets with One Member . . . . . . . . . . . . . . . . . . . . . 176 14.2.7. Extensibility . . . . . . . . . . . . . . . . . . . 177 15. ABNF Normative Definition . . . . . . . . . . . . . . . . . . 177 16. XML Schemas . . . . . . . . . . . . . . . . . . . . . . . . . 191 16.1. NLSML Schema Definition . . . . . . . . . . . . . . . . 192 16.2. Enrollment Results Schema Definition . . . . . . . . . . 193 16.3. Verification Results Schema Definition . . . . . . . . . 194 17. References . . . . . . . . . . . . . . . . . . . . . . . . . 197 17.1. Normative References . . . . . . . . . . . . . . . . . . 197 17.2. Informative References . . . . . . . . . . . . . . . . . 199 Appendix A. Contributors . . . . . . . . . . . . . . . . . . . . 200 Appendix B. Acknowledgements . . . . . . . . . . . . . . . . . . 201 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 202 Intellectual Property and Copyright Statements . . . . . . . . . 203 Shanmugham & Burnett Expires December 11, 2006 [Page 7] Internet-Draft MRCPv2 June 2006 1. Introduction The MRCPv2 protocol is designed to allow a client device to control media processing resources on the network. Some of these media processing resources include speech recognition engines, speech synthesis engines, speaker verification and speaker identification engines. MRCPv2 enables the implementation of distributed Interactive Voice Response platforms using VoiceXML [12] browsers or other client applications while maintaining separate back-end speech processing capabilities on specialized speech processing servers. MRCPv2 is based on the earlier Media Resource Control Protocol (MRCP) [30] developed jointly by Cisco Systems, Inc., Nuance Communications, and Speechworks Inc. The protocol requirements of SPEECHSC[1] dictate that the solution be capable of reaching a media processing server and setting up communication channels to the media resources, and sending and receiving control messages and media streams to/from the server. The Session Initiation Protocol (SIP) [3] meets these requirements. MRCPv2 leverages these capabilities by building upon SIP and the Session Description Protocol (SDP) [4]. MRCPv2 uses SIP to setup and tear down media and control sessions with the server. In addition, the client can use a SIP re-INVITE method(an INVITE dialog sent within an existing SIP Session) to change the characteristics of these media and control session while maintaining the SIP dialog between the client and server. SDP is used to describe the parameters of the media sessions associated with that dialog. It is mandatory to support SIP as the session establishment protocol to ensure interoperability. Other protocols can be used for session establishment by prior agreement. This document only describes the use of SIP and SDP MRCPv2 uses SIP and SDP to create the client/server dialog and set up the media channels to the server. It also uses SIP and SDP to establish MRCPv2 control sessions between the client and the server for each media processing resource required for that dialog. The MRCPv2 protocol exchange between the client and the media resource is carried on that control session. MRCPv2 protocol exchanges do not change the state of the SIP dialog, the media sessions, or other parameters of the dialog initiated via SIP. It controls and affects the state of the media processing resource associated with the MRCPv2 session(s). MRCPv2 defines the messages to control the different media processing resources and the state machines required to guide their operation. It also describes how these messages are carried over a transport layer protocol such as TCP or TLS (Note: SCTP is a viable transport for MRCPv2 as well, but the mapping onto SCTP is not described in Shanmugham & Burnett Expires December 11, 2006 [Page 8] Internet-Draft MRCPv2 June 2006 this specification). 2. Document Conventions RFC2119 [5] provides the interpretations for the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" found in this document. Since many of the definitions and syntax are identical to HTTP/1.1 (RFC2616 [6]), this specification refers to the section where they are defined rather than copying it. For brevity, [HX.Y] is to be taken to refer to Section X.Y of RFC2616. All the mechanisms specified in this document are described in both prose and an augmented Backus-Naur form (ABNF [9]). The complete message format in ABNF form is provided in Section 15 and is the normative format definition. 2.1. Definitions Media Resource An entity on the speech processing server that can be controlled through the MRCPv2 protocol. MRCP Server Aggregate of one or more "Media Resource" entities on a Server, exposed through the MRCPv2 protocol ("Server" for short). MRCP Client An entity controlling one or more Media Resources through the MRCPv2 protocol ("Client" for short). DTMF Dual Tone Multi-Frequency; a method of transmitting key presses in-band, either as actual tones (Q.23 [28]) or as named tone events (RFC2833 [29]). Hotword Mode A mode of speech recognition where a stream of utterances is evaluated for match against a small set of command words. This is generally employed to either trigger some action, or to control the subsequent grammar to be used for further recognition 3. Architecture A system using MRCPv2 consists of a client that requires the generation and/or consumption of media streams and a media resource Shanmugham & Burnett Expires December 11, 2006 [Page 9] Internet-Draft MRCPv2 June 2006 server that has the resources or "engines" to process these streams as input or generate these streams as output. The client uses SIP and SDP to establish an MRCPv2 control channel with the server to use its media processing resources. MRCPv2 servers are addressed using SIP URIs. The session management protocol (SIP) uses SDP with the offer/answer model described in RFC3264 [7] to set up the MRCPv2 control channels and describe their characteristics. A separate MRCPv2 session is needed to control each of the media processing resources associated with the SIP dialog between the client and server. Within a SIP dialog, the individual resource control channels for the different resources are added or removed through SDP offer/answer carried in a SIP re-INVITE transaction. The server, through the SDP exchange, provides the client with an unambiguous channel identifier and a TCP port number. The client MAY then open a new TCP connection with the server using this port number. Multiple MRCPv2 channels can share a TCP connection between the client and the server. All MRCPv2 messages exchanged between the client and the server carry the specified channel identifier that the server MUST ensure is unambiguous among all MRCPv2 control channels that are active on that server. The client uses this channel identifier to indicate the media processing resource associated with that channel. The session management protocol (SIP) also establishes the media sessions between the client (or other source/sink of media) and the MRCPv2 server using SDP m-lines. One or more media processing resources may share a media session under a SIP session, or each media processing resource may have its own media session. Shanmugham & Burnett Expires December 11, 2006 [Page 10] Internet-Draft MRCPv2 June 2006 MRCPv2 client MRCPv2 Media Resource Server |--------------------| |-----------------------------| ||------------------|| ||---------------------------|| || Application Layer|| || TTS | ASR | SV | SI || ||------------------|| ||Engine|Engine|Engine|Engine|| ||Media Resource API|| ||---------------------------|| ||------------------|| || Media Resource Management || || SIP | MRCPv2 || ||---------------------------|| ||Stack | || || SIP | MRCPv2 || || | || || Stack | || ||------------------|| ||---------------------------|| || TCP/IP Stack ||----MRCPv2---|| TCP/IP Stack || || || || || ||------------------||-----SIP-----||---------------------------|| |--------------------| |-----------------------------| | / SIP / | / |-------------------| RTP | | / | Media Source/Sink |-------------/ | | |-------------------| Figure 1: Architectural Diagram 3.1. MRCPv2 Media Resource Types An MRCPv2 server may offer one or more of the following media processing resources to its clients. Basic Synthesizer A speech synthesizer resource with very limited capabilities, that can generate its media stream exclusively from concatenated audio clips. The speech data is described using a limited subset of SSML [25] elements. A basic synthesizer MUST support the SSML tags ,