Network Working Group E. Burger
Internet Draft SnowShore Networks, Inc.
Document: draft-ietf-speechsc-reqts-01.txt D. Oran
Category: Informational Cisco Systems, Inc.
Expires April 2003 October 8, 2002
Requirements for Distributed Control of ASR, SI/SV and TTS Resources
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 [1].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts. Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet- Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
1.
Abstract
This document outlines the needs and requirements for a protocol to
control distributed speech processing of audio streams. By speech
processing, this document specifically means automatic speech
recognition, speaker recognition (which includes both speaker
identification and speaker verification) and text-to-speech. Other
IETF protocols, such as SIP and RTSP, address rendezvous and control
for generalized media streams. However, speech processing presents
additional requirements that none of the extant IETF protocols
address.
Discussion of this and related documents is on the speechsc mailing
list. To subscribe, send the message "subscribe speechsc" to
speechsc-request@ietf.org. The public archive is at
http://www.ietf.org/mail-
archive/workinggroups/speechsc/current/maillist.html.
2.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC-2119 [2].
FORMATTING NOTE: Notes, such at this one, provide additional,
nonessential information that the reader may skip without missing
anything essential. The primary purpose of these non-essential
notes is to convey information about the rationale of this document,
Burger & Oran Informational ? Expires August 2002 1
Distributed Media Control Requirements February 2002
or to place this document in the proper historical or evolutionary
context. Readers whose sole purpose is to construct a conformant
implementation may skip such information. However, it may be of use
to those who wish to understand why we made certain design choices.
OPEN ISSUES: This document highlights questions that are, as yet,
undecided as "OPEN ISSUES".
3.
Introduction
There are multiple IETF protocols for establishment and termination
of media sessions (SIP[5]), low-level media control (MGCP[6] and
MEGACO[7]), and media record and playback (RTSP[8]). This document
focuses on requirements for one or more protocols to support the
control of network elements that perform Automated Speech
Recognition (ASR), speaker identification or verification (SI/SV),
and rendering text into audio, a.k.a. Text-to-Speech (TTS). Many
multimedia applications can benefit from having automatic speech
recognition (ASR) and text-to-speech (TTS) processing available as a
distributed, network resource. This requirements document limits
its focus on the distributed control of ASR, SI/SV and TTS servers.
There are a broad range of systems which can benefit from a unified
approach to control of TTS, ASR, and SI/SV. These include
environments such as VoIP gateways to the PSTN, IP Telephones, and
wireless mobile devices who obtain speech services via servers on
the network.
To date, there are a number of proprietary ASR and TTS API's, as
well as two IETF drafts that address this problem [9] [10].
However, there are serious deficiencies to the existing drafts. In
particular, they mix the semantics of existing protocols yet are
close enough to other protocols as to be confusing to the
implementer.
This document sets forth requirements for protocols to support
distributed speech processing of audio streams. For simplicity, and
to remove confusion with existing protocol proposals, this document
presents the requirements as being for a "new protocol" that
addresses the distributed control of speech resources It refers to
such a protocol as "SPEECHSC", for Speech Services Control Protocol.
Burger & Oran Informational ? Expires August 2002 2
Distributed Media Control Requirements February 2002
4.
SPEECHSC Framework
The following is the SPEECHSC framework for speech processing.
+-------------+
| Application |
| Server |\
+-------------+ \ SPEECHSC
SIP or whatever / \
/ \
+------------+ / \ +--------+
| Media |/ SPEECHSC \---| ASR |
| Processing |-------------------------| and/or |
RTP | Entity | RTP | TTS |
=====| |=========================| Server |
+------------+ +--------+
The "Media Processing Entity" is a network element that processes
media. The "Application Server" is a network element that instructs
the Media Processing Entity on what transformations to make to the
media stream. The "ASR and/or TTS Server" is a network element that
either generates a RTP stream based on text input (TTS) or returns
speech recognition results in response to an RTP stream as input
(ASR). Either the Media Processing Entity or the Application Server
may control the ASR or TTS Server using SPEECHSC as a control
protocol.
Physical embodiments of the entities can reside in one physical
instance per entity, or some combination of entities. For example,
a VoiceXML [11] Gateway may combine the ASR and TTS functions on the
same platform as the Media Processing Entity. Note that VoiceXML
Gateways themselves are outside the scope of this protocol.
Likewise, one can combine the Application Server and Media
Processing Entity, as would be the case in an interactive voice
response (IVR) platform.
One can also decompose the Media Processing Entity into an entity
that controls media endpoints and entities that process media
directly. Such would be the case with a decomposed gateway using
MGCP or megaco. However, this decomposition is again orthogonal to
the scope of SPEECHSC.
5.
General Requirements
5.1.
Reuse Existing Protocols
To the extent feasible, the SPEECHSC framework SHOULD use existing
protocols.
5.2.
Maintain Existing Protocol Integrity
Burger & Oran Informational ? Expires August 2002 3
Distributed Media Control Requirements February 2002
In meeting requirement 5.1, the SPEECHSC framework MUST NOT redefine
the semantics of an existing protocol. Said differently, we will not
break existing protocols or cause backward compatibility problems.
5.3.
Avoid Duplicating Existing Protocols
To the extent feasible, SPEECHSC SHOULD NOT duplicate the
functionality of existing protocols. For example, SIP with msuri
[12] and RTSP already define how to request playback of audio.
The focus of SPEECHSC is new functionality not addressed by existing
protocols or extending existing protocols within the strictures of
requirement 5.2. Where an existing protocol can be gracefully
extended to support SPEECHSC requirements, such extensions are
acceptable alternatives for meeting the requirements.
5.4.
Protocol efficiency
The SPEECHSC framework SHOULD employ protocol elements known to
result in efficient operation. Techniques to be considered include:
- Re-use of transport connections across sessions
- Piggybacking of responses on requests in the reverse
direction
- Caching of state across requests
5.5.
Explicit invocation of services
The SPEECHSC framework MUST be compliant with the IAB OPES[5]
framework. The applicability of the SPEECHSC protocol will therefore
be specified as occurring between clients and servers at least one
of which is operating directly on behalf of the user requesting the
service.
5.6.
Server Location and Load Balancing
To the extent feasible, the SPEECHSC framework SHOULD exploit
existing schemes for performing service location and load balancing,
such as the Service Location Protocol[13] or DNS SRV records[14].
Where such facilities are not deemed adequate, the SPEECHSC
framework MAY define additional load balancing techniques.
5.7.
Simultaneous services
The SPEECHSC framework MUST permit multiple services to operate on a
single media stream so that either the same or different servers may
be performing speech recognition, speaker identification or
verification, etc. in parallel.
5.8.
Multiple media sessions
The SPEECHSC framework MUST allow a 1:N mapping between session and
RTP channels. For example, a single session may include an outbound
RTP channel for TTS, an inbound for ASR and a different inbound for
SI/SV (e.g. if processed by different elements on the Media Resource
Burger & Oran Informational ? Expires August 2002 4
Distributed Media Control Requirements February 2002
Element). Note: All of these can be described via SDP, so if SDP is
utilized for media channel description, this requirement is met ?for
free?.
6.
TTS Requirements
6.1.
Requesting Text Playback
The SPEECHSC framework MUST allow a Media Processing Entity or
Application Server, using a control protocol, to request the TTS
Server to playback text as voice in an RTP stream.
6.2.
Text Formats
6.2.1.
Plain Text
The TTS Server MUST support the reading of plain text. For reading
plain text, the language and voicing MAY be indicated via session
parameters. For finer control over such properties, use of SSML
rather than plain text provides the necessary capabilities.
6.2.2.
SSML
The TTS Server SHOULD support the reading of SSML [3] text.
6.2.3.
Text in Control Channel
The TTS Server MUST accept text over the SPEECHSC connection for
reading over the RTP connection. The server MUST accept text either
?by value? (embedded in the protocol), or ?by reference? (by de-
referencing a URI embedded in the protocol).
6.2.4.
Document Type Indication
The SPEECHSC framework MUST be capable of explicitly indicating the
document type of the text to be processed, as opposed to forcing the
server to infer the content by other means.
6.3.
Control Channel
The SPEECHSC framework MUST be capable of establishing the control
channel between the client and server on a per-session basis, where
a session is loosely defined to be associated with a single ?call?
or ?dialog?. The protocol SHOULD be capable of maintaining a long-
lived control channel for multiple sessions serially, and MAY be
capable of shorter time horizons as well, including as short as for
the processing of a single utterance.
6.4.
Playback Controls
The TTS Server SHOULD support, and the SPEECHSC framework MUST
support the specification of, "VCR Controls":
Burger & Oran Informational ? Expires August 2002 5
Distributed Media Control Requirements February 2002
- The ability to jump in time to the location of a specific
marker.
- The ability to jump in time, forwards or backwards, by a
specified amount of time. Valid time units MUST include
seconds, words, paragraphs, sentences, and markers.
- The ability to increase and decrease playout speed.
- The ability to fast-forward and fast-rewind the audio, where
snippets of audio are played as the server moves forwards or
backwards in time.
- The ability to pause and resume playout.
- The ability to increase and decrease playout volume.
6.5.
Session Parameters
The SPEECHSC framework must support the specification of session
parameters, such as language, prosody and voicing.
6.6.
Speech Markers
The SPEECHSC framework MUST accommodate speech markers, with
capability at least as flexible as that provided in SSML[3]. The
framework MUST further provide an efficient mechanism for reporting
that a marker has been reached during playout.
7.
ASR Requirements
7.1.
Requesting Automatic Speech Recognition
The SPEECHSC framework MUST allow a Media Processing Entity or
Application Server to request the ASR Server to perform automatic
speech recognition on an RTP stream, returning the results over
SPEECHSC.
7.2.
XML
The ASR Server MUST support the XML specification for speech
recognition [4].
7.3.
Grammar Requirements
7.3.1.
Grammar Specification
The ASR Server MUST accept grammar specifications either ?by value?
(embedded in the protocol), or ?by reference? (by de-referencing a
URI embedded in the protocol). The latter MUST allow the indication
of a grammar already known to, or otherwise ?built in? to the
server. Servers SHOULD be able to store and later retrieve by
reference large grammars which were originally supplied by the
client.
7.3.2.
Explicit Indication of Grammar Format
Burger & Oran Informational ? Expires August 2002 6
Distributed Media Control Requirements February 2002
The SPEECHSC framework protocol MUST be able to explicitly convey
the grammar format in which the grammar is encoded and MUST be
extensible to allow for conveying new grammar formats as they are
defined.
7.3.3.
Grammar Sharing
The ASR Server SHOULD support sharing grammars across sessions.
This supports applications with large grammars for which it is
unrealistic to dynamically load. An example is a city-country
grammar for a weather service.
7.4.
Session Parameters
The SPEECHSC framework MUST accommodate at a minimum all of the
protocol parameters currently defined in MRCP[7]. In addition there
SHOULD be a capability to reset parameters within a session.
7.5.
Input Capture
The SPEECHSC framework MUST support a method directing the ASR
Server to capture the input media stream for later analysis and
tuning of the ASR engine. The ASR Server SHOULD support this
capability.
8.
Speaker Identification and Verification Requirements
8.1.
Requesting SI/SV
The SPEECHSC framework MUST allow a Media Processing Entity to
request the SI/SV Server to perform speaker identification or
verification on an RTP stream, returning the results over SPEECHSC.
8.2.
Identifiers for SI/SV
The SPEECHSC framework MUST accommodate an identifier for each
verification resource and permit control of that resource by ID,
because voiceprint format and contents are vendor specific.
8.3.
State for multiple utterances
The SPEECHSC framework MUST work with SI/SV servers which maintain
state to handle multi-utterance verification.
8.4.
Input Capture
The SPEECHSC framework, and SI/SV Server MUST support a method for
capturing the input media stream for later analysis and tuning of
the SI/SV engine. The ASR Server SHOULD support this capability.
Burger & Oran Informational ? Expires August 2002 7
Distributed Media Control Requirements February 2002
8.5.
SI/SV functional extensibility
The SPEECHSC framework SHOULD be extensible to additional functions
associated with SI/SV, such as prompting, utterance verification,
and retraining.
9.
Duplexing and Parallel Operation Requirements
One very important requirement for an interactive speech-driven
system is that user perception of the quality of the interaction
depends strongly on the ability of the user to interrupt a prompt or
rendered TTS with speech. Interrupting, or barging, the speech
output requires more than energy detection from the user's
direction. Many advanced systems halt the media towards the user by
employing the ASR engine to decide if an utterance is likely to be
real speech, as opposed to a cough, for example.
9.1.1.
Full Duplex operation
To achieve low latency between utterance detection and halting of
playback, many implementations combine the speaking and ASR
functions. The SPEECHSC framework MUST support such full-duplex
implementations.
9.1.2.
Multiple services in parallel
Good spoken user interfaces typically depend upon the ease with
which the user can accomplish his or her task. When making use of
Speaker Identification or Verification technologies, user interface
improvements often come from the combination of the different
technologies: simultaneous identity claim and verification (on the
same utterance), simultaneous knowledge and voice verification
(using ASR and verification simultaneously). Using ASR and
verification on the same utterance is in fact the only way to
support rolling or dynamically-generated challenge phrases (e.g.,
"say 51723"). The SPEECHSC framework MUST support such parallel
service implementations.
10.
Additional Considerations (non-normative)
The framework assumes that SDP will be used to describe media
sessions and streams. The framework further assumes RTP carriage of
media, however since SDP can be used to describe other media
transport schemes (e.g. ATM) these could be used if they provide the
necessary elements (e.g. explicit timestamps).
The working group will not be defining distributed speech
recognition methods (DSR), as exemplified by the ETSI Aurora
project. The working group will not be recreating functionality
available in other protocols, such as SIP or SDP.
TTS looks very much like playing back a file. Extending RTSP looks
promising for when one requires VCR controls or markers in the text
Burger & Oran Informational ? Expires August 2002 8
Distributed Media Control Requirements February 2002
to be spoken. When one does not require VCR controls, SIP in a
framework such as Network Announcements [10] works directly without
modification.
ASR has an entirely different set of characteristics. For barge-in
support, ASR requires real-time return of intermediate results.
Barring the discovery of a good reuse model for an existing
protocol, this will most likely become the focus of SPEECHSC.
11.
Security Considerations
Protocols relating to speech processing must take security into
account. This is particularly important as popular uses for TTS
include reading financial information. Likewise, popular uses for
ASR include executing financial transactions and shopping.
We envision that rather than providing application-specific security
mechanisms in SPEECHSC itself, the resulting protocol will employ
security machinery of either containing protocols or the transport
on which it runs. For example, we will consider solutions such as
using TLS for securing the control channel, and SRTP for securing
the media channel. Third-part dependencies necessitating transitive
trust will be minimized or explicitly dealt with through the
authentication and authorization aspects of the protocol design.
In addition to the security machinery needed by the protocol itself,
there are considerations for the implementation and deployment of
the clients and servers themselves. For example, speaker verifica-
tion and identification employs voiceprints whose privacy and
integrity must be maintained. While strictly speaking out of scope
of the protocol itself, such considerations will be carefully
considered and accommodated during protocol design, and will be
called out as part of the applicability statement accompanying the
protocol specification(s).
12.
Normative References
1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP
9, RFC 2026, October 1996.
2 Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997
3 World Wide Web Consortium, "Speech Synthesis Markup Language
Specification for the Speech Interface Framework", W3C Working
Draft 5, ,
April 2002, work in progress
Burger & Oran Informational ? Expires August 2002 9
Distributed Media Control Requirements February 2002
4 World Wide Web Consortium, "Speech Recognition Grammar
Specification Version 1.0", W3C Candidate Recommendation,
, June
2002, work in progress
5 Floyd, S., Daigle, L., ?IAB Architectural and Policy
Considerations for Open Pluggable Edge Services,? RFC3238,
January 2002
13.
Informative References
5 Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A.,
Peterson, J., Sparks, R., Handley, H., Schooler, E., "SIP:
Session Initiation Protocol", RFC 3261, June 2002.
6 Arango, M., Dugan, A., Elliott, I., Huitema, C., and Pickett, S.,
"Media Gateway Control Protocol (MGCP) Version 1.0", RFC 2705,
October 1999
7 Cuervo, F., Greene, N., Rayhan, A., Huitema, C., Rosen, B., and
Segers, J., "Megaco Protocol Version 1.0", RFC 3015, November
2000
8 Schulzrinne, H., Rao, A., and Lanphier, R., "Real Time Streaming
Protocol (RTSP)", RFC 2326, April 1998
9 Shanmugham, S., Monaco, P., and B. Eberman, "MRCP: Media Resource
Control Protocol", draft-shanmugham-mrcp-02.txt, July 2002, work
in progress
10 Robinson, F., Marquette, B., and R. Hernandez, "Using Media
Resource Control Protocol with SIP", draft-robinson-mrcp-sip-
00.txt, January 2002, work in progress
11 World Wide Web Consortium, "Voice Extensible Markup Language
(VoiceXML) Version 2.0", W3C Working Draft,
,
April 2002, work in progress
12 Van Dyke, J., Burger, E., Spitzer, A., O'Connor, W., "Basic
Network Media Services with SIP", draft-burger-sipping-netann-
02.txt, June 2002, work in progress
13 Guttman, E., Perkins, C., Veizades, J., Day, M. , "Service
Location Protocol, Version 2,? RFC 2608, June 1999.
14 Gulbrandson, A, Vixie, P., Esibov, L., ?A DNS RR for specifying
the location of services (DNS SRV)?, RFC2782, February 2000.
Burger & Oran Informational ? Expires August 2002 10
Distributed Media Control Requirements February 2002
14.
Acknowledgments
Stephane Maes, Sarvi Shanmughan, Brian Eberman, Dan Burnett, and
Brian Wyld all made significant contributions of requirements and
proposed text for capturing them.
15.
Author's Addresses
Eric W. Burger
SnowShore Networks, Inc.
Chelmsford, MA
USA
Email: eburger@snowshore.com
David R. Oran
Cisco Systems, Inc.
Acton, MA
USA
Email: oran@cisco.com
16.
Change Log
From version draft-burger-mrcp-reqts-00 to version draft-burger-
speechsc-reqts-00:
- draft name changed per area director advice
- added speaker verification to the areas addressed, including
speaker verification requirements, per Dan Burnet?s
presentation at the Minneapolis BoF (see minutes).
- based on mailing list discussion, added requirement to handle
both ?by value? and ?by reference? data. This is both for TTS
to be played out and grammar(s) to be applied to ASR.
- Based on discussion at the BoF in Minneapolis, added a
requirement concerning the use of load balancing schemes,
including those based on SRVLOC, SRV.
- Added a requirement for OPES compliance, per a discussion
with Sally Floyd as IAB observer for the BoF.
From version draft-burger-speechsc-reqts-00 to version draft-ietf-
speechsc-reqts-00:
- Changed ?SV? to ?SR? and ?speaker verification? to ?speaker
recognition? everywhere
- Replaced SRCP with SPEECHSC everywhere
- Minor edits including mailing list name change, temporary
notes removed,
- All agreements reached at the IETF 54 WG meeting, confirmed
by mailing list discussion, up through 8/10/02 have been
integrated
- Improved requirement on VCR controls as suggested by Dan
Burnett and Sarvi Shanmughan
- Text describing dual-mode requirements for ASR and SR by Dan
Burnett added.
Burger & Oran Informational ? Expires August 2002 11
Distributed Media Control Requirements February 2002
- Suggested change to framework figure made by Rajiv
Dharmadhikari incorporated
- Updated references to most recent versions
From version draft-ietf-speechsc-reqts-00 to version draft-ietf-
speechsc-reqts-01.txt:
- Adopted Rajiv D.'s wording clarification to the TTS & ASR
requirements to allow control to come from either a separate
Application Server, or a combined server with a Media
Processing entity.
- Reorganized references into separate normative and
informative sections as requested by Scott Bradner
- Added numbering for requirements in sections that were not
previously numbered. This necessitated a bit of text
shuffling to group related requirements more closely
together.
- Added a paragraph to the introduction to emphasize the wide
variety of applications of the speechsc framework and
explicitly call out wireless mobile devices, IP phones, and
PSTN VoIP gateways.
- During WGLC, the use of the term "speaker recognition" to
cover both speaker identification and speaker verification
was questioned. In addition some WG participants felt that
there should be separate requirements for each, while others
argued that the differences, while affecting the structure of
the application, did not affect the requirements for the
protocol in any substantive way. There were views that the
existing terminology was common in the industry and hence
should not be changed, and sentiments for a variety of other
solutions. The best compromise seemed to be to continue to
group the requirements together, but point out where there
may be subtle differences affecting applications. It also
seemed prudent to keep the identification/verification
distinction in the terminology, and hence the document uses
the acronym SI/SV rather than SR when talking about both
together.
- added a requirement in section 5 for 1:N mapping of control
to media channels, but pointed out that if SDP is used this
comes for free.
- Changed the input capture requirement from SHOULD to MUST,
but made implementation by the server a SHOULD.
- added a SHOULD requirement for protocol efficiency with re-
use of transport connections as one of a set of examples.
Burger & Oran Informational ? Expires August 2002 12
Distributed Media Control Requirements February 2002
- noted in section 10 that while RTP is assumed, the framework
applies to other media carriage schemes that can be described
by SDP, as long as they have the right features.
Burger & Oran Informational ? Expires August 2002 13
Distributed Media Control Requirements February 2002
Full Copyright Statement
Copyright (C) The Internet Society (2002). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns. This
document and the information contained herein is provided on an "AS
IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK
FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL
NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY
OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
The Internet Society currently provides funding for the RFC Editor
function.
Burger & Oran Informational ? Expires August 2002 14