INTERNET-DRAFT A. Soloviev draft-avsolov-dtpdia-05.txt Petrozavodsk State University Expires: March 1, 2005 September 1, 2004 Data Transfer Protocol for Distributed Information Acquisition (DTP/DIA) Status of This Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC 2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on March 1, 2005. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. Abstract The described in this memo protocol was developed for application in some distributed information measurement systems (IMS) at any stage of communication. It was designed for transmission of measured data from the measuring devices to the processing and storing equipment. Also it does support a common identification of the devices in distributed IMS. Due to its simplicity it can be easily implemented in firmware of any digital measuring device. Soloviev Expires Match 1, 2005 [Page 1] Internet-Draft DTP/DIA September 2004 1. Introduction The Data Transfer Protocol for Distributed Information Acquisition (DTP/DIA) was developed for application in distributed information measurement systems (IMS). DTP/DIA provides functionality of the presentation and application layers of the OSI Reference Model for the specified purposes. 1.1. Concept of Distributed IMS IMS stands for a bundle of software and hardware that performs acquisition, processing, storing, and presentation of measured data. The systems with control function are out of scope. The simplest IMS consists of some measuring device connected to computer by means of some instrument interface (such as IEEE 488 (GPIB) or EIA/RS 232). A typical measuring device contains a sensor and a microcontroller that performs initial data acquisition and processing. In such a system the majority of IMS functions are given to a computer. More complicated systems can be developed on the basis of CAMAC or VXI. But projects with large amount of investigated objects at far distances from each other require to install distributed IMS. Some nodes of distributed IMS should perform data transmission to nodes that process and store data. Sometimes this communication can be done by the measuring device itself if it is rather sophisticated, or this function may be performed by a computer. That is we deal with two kinds of data channels: local (1) and network (2). +---------+ +-----------------------+ |measuring|==============(2)============| | computer ............| | device | ||==|(collector): database | +---------+ (1)-local (instrument) bus || +-----------------------+ (2)-network channel || +-----------------------+ +---------+ ||==| computer ............| |measuring| +--------------+ || |(collector): web-server| | device |--(1)--| | || +-----------------------+ +---------+ | computer | || +-----------------------+ +---------+ |(retranslator)| ||==| computer :calculating| |measuring|--(1)--| |==(2)=| |(collector): software | | device | +--------------+ +-----------------------+ +---------+ In most cases the nodes of distributed IMS are considered to be the components of the open systems. So the OSI Reference Model can be applied to their communication channels. The upper layers of such systems are poorly standardized due to wide range of applications of IMS. The most of existing standards are vendor-specific. That's why it is necessary to introduce a simple protocol, suitable for both Soloviev Expires Match 1, 2005 [Page 2] Internet-Draft DTP/DIA September 2004 kinds of data channels. DTP/DIA matches these requirements. 1.2. Remark about Terms We avoid using the terms "client" and "server" in description of IMS. Instead we use the term "data source" for the object which transmits measured data and "data collector" for the node which receives measured data. If the node performs both reception and retransmission it will be named "retranslator". Obviously, in this terminology the measuring devices are "data sources", but a single computer could be either end-point "data collector" or "retranslator". The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 2. General Features The protocol describes data transmission in small stand-alone packets. This feature allows using DTP/DIA over both streamed (connection-oriented) and message-oriented channels. DTP/DIA provides several data presentation forms. Each form corresponds to a specific packet type. Some forms help to avoid implementation of floating point arithmetics in firmware of the device. There are forms that allow to transmit service or control information. DTP/DIA allows transmission of measured data accompanied with measurement error and unit measure. In distributed IMS it is very important to distinguish the measuring devices from each other at the application layer. So the protocol includes identification mechanism. This feature helps to transmit data from several data sources over a single communication channel. For example, one can use a device with several sensors (such device is represented as multiple data sources). We suppose to apply DTP/DIA for both network and local channels. Local channels of some types don't provide reliable data delivery. To avoid unreliability of local interfaces DTP/DIA has the following primitive features: detection of packet start (packet leading sequence), check sum and time stamp. Soloviev Expires Match 1, 2005 [Page 3] Internet-Draft DTP/DIA September 2004 3. DTP/DIA Packet Format DTP/DIA packet contains the fixed-sized Header block, the Data field and several optional fields. Here and further the octets numbering starts from 0. Header block contains packet leading sequence, version field (VERS), flags field (L, T, and U bits), Data Source Identifier (ID.1, ID.2, and ID.3 fields), size field (SIZE), packet type field (TYPE), and device vendor specific data (DEVINFO). The size of the Header block is 8 octets. This block is REQUIRED. The DATA filed may contain measuring information, unit measure mark, accuracy information, it may represent List of Inquiring Identifiers, or some special information. The DATA field may be followed by TIMESTAMP and CHECKSUM. A simple measuring device must produce packets which contains at least Header block and a simple form of the Data field. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0 0 1 0 0 1 0|0 0 1 0 1 0 1 0| VERS |L|T|U|R| ID.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ID.2 | ID.3 | SIZE | TYPE | DEVINFO | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+- . . . DATA . . . -+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TIMESTAMP | CHECKSUM | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3.1. Packet Leading Sequence The first two octets of DTP/DIA packet contain values 0x49 and 0x54. These values must be used by software to detect the origin of DTP/DIA packet. Collector software must ignore any data not started with the specified packet leading sequence. 3.2. Version Field Current version code is 0. Soloviev Expires Match 1, 2005 [Page 4] Internet-Draft DTP/DIA September 2004 3.3. Flags Field The 20th bit (L) determines the kind of byte order of multi-byte fields in the packet. 1 corresponds to little-endian format, 0 corresponds to big-endian format. It's up to IMS developer to choose byte order format. Obviously, it depends on what kind of microcontroller was used in the measuring device. For example, Intel's MCS 96/296 uses little-endian byte order, but Motorola's MC68xxx uses big-endian byte order. The 21st bit (T) determines whether TIMESTAMP field has to be ignored or not (see Section 3.10). If this bit is set (T=1) TIMESTAMP must be ignored, otherwise it requires a valid value inside. The 22nd bit (U) determines whether UTF-8 [10] encoding should be used for representation of textual information. If this bit is set (U=1) any text in the DATA field must be encoded with UTF-8, otherwise text is represented as common ASCII. The 23rd bit is reserved for future use and must be 0. 3.4. Data Source Identifier To distinguish data from various data sources DTP/DIA packet contains Data Source Identifier. To introduce a bit of structurization in distributed IMS the Identifier was divided into three parts: ID.1, ID.2, and ID.3. They correspond to high, middle, and low levels of 3-level hierarchy of IMS. The notation "AAA/BBB/CCC" will be used to specify Data Source Identifier (where AAA - ID.1, BBB - ID.2, CCC - ID.3). It's not recommended to assign a fixed identifier for data source by device vendor. Developer of IMS should have an opportunity to change Data Source Identifier, for example, by means of special software or DIP-switches. Device vendor may apply Data Source Identification Procedure for assigning identifiers as described in Section 4. Identifiers "0/0/0" and "255/255/255" are reserved for Data Source Identification Procedure (see Section 4). Every retranslator SHOULD keep Data Source Identifier untouched. 3.5. Size of the Packet The size of the packet specifies amount of 32-bit words occupied by the packet. The minimal size of DTP/DIA packet is 12 bytes (SIZE=3). The maximal size of DTP/DIA packet is 60 bytes (SIZE=15). Soloviev Expires Match 1, 2005 [Page 5] Internet-Draft DTP/DIA September 2004 The values 0, 1, and 2 are illegal. Collector software should scan 7 bytes back for another packet leading sequence or discard received bytes. 3.6. Packet Types The following packet types are declared: TYPE_FLOAT (TYPE=0), TYPE_INT1 (TYPE=1), TYPE_INT2 (TYPE=2), TYPE_INT3 (TYPE=3), TYPE_INFO (TYPE=14), TYPE_SPEC (TYPE=15). Other values are reserved for future use. Collector software should ignore the packets of reserved types. TYPE_FLOAT, TYPE_INT1, TYPE_INT2, and TYPE_INT3 specify different data presentation forms (as described in Section 3.8). TYPE_INFO packets are designed to provide additional textual information about data source. Octets from the 8th to the end of the packet (except for the last 32-bit word) are filled with some text (firmware version, copyrights, vendor information etc). Encoding of this text depends on flag U. Such text must be terminated by at least one octet zero and padding bytes must contain zeros as well. Maximal length of this text without trailing zeros is 47 bytes. Collector software may silently ignore such packets. TYPE_SPEC packets have a special purpose in the Data Source Identification Procedure (see Section 4). When this packet is used in Data Source Identification Procedure, Measured Data block must be filled with zeros. In other cases Measured Data block may contain some device state information when the packet was issued by data source, or it may contain some control information when the packet was issued by collector. These values are vendor specific. 3.7. Device Vendor Specific Data The DEVINFO field contains device vendor specific information. This value MUST NOT influence on interpretation of MEASURED DATA. It is recommended to code the model of the device in this field, so the whole Header block might be hard-coded in the firmware of the device. 3.8. DATA Field The content of the DATA field is determined by the packet type. The size of this field must be devisible by 4. Padding bytes must contain zeroes. TYPE_FLOAT packet (TYPE=0) The DATA field contains a certain value of physical quantity, represented as 32-bit floating point number ("single" in terms of IEEE 754). These bits contain a sign bit (S), 8 bits of exponent Soloviev Expires Match 1, 2005 [Page 6] Internet-Draft DTP/DIA September 2004 (E), and 23 bits of mantissa's fraction (M1...M23). Initial reference value is calculated in such a way (see details in [6]): (-1)^S * 2^(E-127) * 1.M1M2M3...M23 Here we use "^" as exponentiating operator. This type supports the initial reference values in the range 1.18E-38... 3.40E+38 (in magnitude) TYPE_INT1 packet (TYPE=1) The DATA field contains multiplied by 10 value of physical quantity, represented as 32-bit signed integer number. This type supports the initial reference values in the range -214,748,364.8 ... 214,748,364.7 TYPE_INT2 packet (TYPE=2) The DATA field contains multiplied by 100 value of physical quantity, represented as 32-bit signed integer number. This type supports the initial reference values in the range -21,474,836.48 ... 21,474,836.47 TYPE_INT3 packet (TYPE=3) The DATA field contains multiplied by 1000 value of physical quantity, represented as 32-bit signed integer number. This type supports the initial reference values in the range -2,147,483.648 ... 2,147,483.647 Byte order of Measured Data block depends on the value of flag L. Retranslator may convert measured data presentation from one form to another. If information about measurement error and unit measure are to be transmitted the measuring information should be followed by unit measure mark and accuracy data. +-----------------+-------+-------+ |UNIT MEASURE MARK| PROB | ERROR | +-----------------+-------+-------+ UNIT MEASURE MARK starts from the 12th octet of the packet. This field is considered to be a text notation of unit measure. It is recommended to place generally used notations, based on [9], into this field. Encoding of this text depends on flag U. Such text must be terminated by at least one octet zero. The size of this field is variable but must be divisible by 4 and padding octets must contain zeros. Maximal length of this text without trailing zeros is 43 bytes. UNIT MEASURE MARK must not be used in TYPE_INFO and TYPE_SPEC packets. UNIT MEASURE MARK is required if accuracy fields are present (but may be filled with zeros). Soloviev Expires Match 1, 2005 [Page 7] Internet-Draft DTP/DIA September 2004 Accuracy fields (PROB and ERROR) start just after the trailing zeros of UNIT MEASURE MARK. PROB offset must be aligned by 4. ERROR field contains the measurement relative error and follows PROB. PROB field contains the probability of physical quantity being outside of the interval determined by measurement relative error (the complement of the reliability to 1). In TYPE_FLOAT packets accuracy fields are represented as 32-bit floating point values and occupy 8 octets. In TYPE_INTx packets they are represented as 16-bit unsigned integer numbers multiplied by 10000 and occupy 4 octets. Byte order of these values depends on the value of flag L. Accuracy fields are OPTIONAL. 3.9. TIMESTAMP To distinguish various packets and so to avoid packet duplication TIMESTAMP may be included into the packet. TIMESTAMP stands for measurement time, represented as 24 least significant bits in seconds, elapsed since 00:00:00 UTC, January 1, 1970. Byte order of TIMESTAMP is determined by bit L in the flags field. If collector receives two packets from one data source with the same TIMESTAMP it should discard one of these packets. This situation may happen not only because of packets duplication but when developer of IMS has implemented the opportunity to correct measured data or to make it more accurate on-the-fly. So collector software may be set up to discard the first packet with the same TIMESTAMP. This field is OPTIONAL. In the case TIMESTAMP and CHECKSUM are absent (SIZE=3), the bit T in the flags field must be set (T=1). If CHECKSUM is included into packet the space for TIMESTAMP is reserved to provide the proper alignment. In the case time stamp information isn't required the bit T must be set (T=1), TIMESTAMP must be filled with zero and will be ignored by collector. If bit T is cleared (T=0) TIMESTAMP contains a valid value and may be treated. 3.10. CHECKSUM CHECKSUM occupies the last octet of the packet. CHECKSUM is the remainder of division of the sum of previous bytes by 256 (modulus of 256). This field is required if the packet is larger than 12 bytes (SIZE>3). The packet with incorrect CHECKSUM must be discarded. 4. Data Source Identification Procedure Data Source Identification Procedure is an optional mechanism that helps sharing a single transport layer connection for data transmission from several data sources. This feature must not be applied to connectionless (message-oriented) channels. So data Soloviev Expires Match 1, 2005 [Page 8] Internet-Draft DTP/DIA September 2004 source can distinguish every collector by a certain transport layer connection. This mechanism consists of two events: "Offer To Identify" and "Device Request". "Offer To Identify" corresponds to TYPE_SPEC packet with ID="255/255/255", sent by data source equipment. This packet may be sent not only just right after connection establishment but at any moment (for example, when hardware configuration of data sources has changed). Expected reaction of collector is "Device Request" packet. If collector ignores "Offer To Identify" (doesn't reply for specified time) the action of data source must be one of the following: - 0..3 retries of "Offer To Identify" then it must break the transport layer connection; - 0..3 retries of "Offer To Identify" then it must start transmission of data packets of data source with the least existing identifier. If collector requests both non-existing and existing data sources the equipment action must be one of the following: - 0..3 retries of "Offer To Identify" then it must break the transport layer connection; - 0..3 retries of "Offer To Identify" then it must ignore the non- existing data sources request; If collector requests non-existing data source only the equipment may break the transport layer connection. "Device Request" corresponds to TYPE_SPEC packet with ID="0/0/0", sent by collector. This packet must contain List of Inquiring Identifiers in the DATA field. The size of this list is variable but divisible by 4 (the size of sequence). The list starts from the 12th octet and lasts to the end of the packet. Each Data Source Identifier in this list occupies 3 last octets in each sequence. Leading byte in each sequence is zero. +0 +1 +2 +3 +----+----+----+----+ | 0 |ID.1|ID.2|ID.3| +----+----+----+----+ List of Inquiring Identifiers may contain up to 11 identifiers of the data sources, expected to send their data through this communication channel. If collector needs to request more than 11 data sources it must distribute their identifiers to several "Device Request" Soloviev Expires Match 1, 2005 [Page 9] Internet-Draft DTP/DIA September 2004 packets. If data source accepts "Device Request" it starts sending data packets. Collector may send "Device Request" packet at any time of communication (not only when it answers to "Offer To Identify"). The difference between stand-alone "Device Request" and the sequence "Offer To Identify" - "Device Request" is that every time data source equipment sends "Offer To Identify" packet it clears the stack of requested identifiers for the corresponding collector, but when data source receives the next "Device Request" it adds identifiers from this packet to current stack of requested identifiers. Also Data Source Identification Procedure may be applied for assigning identifiers. In this case the first identifier requested by collector should be used by data source as its own Data Source Identifier. Retranslator may act as collector or as data source in this Data Source Identification Procedure. 5. Protocol Number IANA has assigned TCP and UDP port number 3489 for DTP/DIA. 6. Security Considerations DTP/DIA is definitely insecure. To produce security based applications developer of IMS should provide authentication and data protection on the transport layer (by means of SSL [5] or TLS [4]. Soloviev Expires Match 1, 2005 [Page 10] Internet-Draft DTP/DIA September 2004 References [1] ANSI, "Coded Character Set - 7-Bit American Standard Code for Information Interchange", ANSI X3.4-1986. [2] Bradner S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, 1997. [3] Bradner S., "The Internet Standards Process - Revision 3", BCP 9, RFC 2026, 1996. [4] Dierks T., Allen C., "The TLS Protocol Version 1.0", RFC 2246, 1999. [5] Frier A., Karlton P., Kocher P., "The SSL 3.0 Protocol", Netscape Communications Corp., 1996. [6] IEEE, "IEEE Standard for Binary Floating-Point Arithmetics", IEEE 754-1985. [7] Postel J., "Transmission Control Protocol", STD 7, RFC 793, 1981. [8] Postel J., "User Datagram Protocol", STD 6, RFC 768, 1980. [9] "Symbols Units and Nomenclature in Physics". Document IUPAP-25, 1987. [10] Yergeau F., "UTF-8, a transformation format of ISO 10646", RFC2279, 1998. Acknowledgements The author gratefully acknowledges the supervision of A. Moschevikin, associate professor, PhD, and valuable advice of A. Korolkov. Soloviev Expires Match 1, 2005 [Page 11] Internet-Draft DTP/DIA September 2004 Author's Address Alexei V. Soloviev Chair of IMS & Physical Electronics Petrozavodsk State University Lenin St. 33 185910 Petrozavodsk, Karelia RUSSIA Phone: +7-8142-711021 E-mail: avsolov @ lab127.karelia.ru Full Copyright Statement Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Soloviev Expires Match 1, 2005 [Page 12] Internet-Draft DTP/DIA September 2004 Appendix: Glossary of Acronyms CAMAC - Computer Automated Measurement And Control EIA/RS - Electronic Industries Association Recommended Standard GPIB - General Purpose Interface Bus IANA - Internet Assigned Numbers Authority IEC/CEI - International Electrotechnical Commission IEEE - Institute of Electrical and Electronics Engineers IETF - Internet Engineering Task Force IMS - Information Measurement System ISO - International Organization for Standardization OSI/RM - Open Systems Interconnection Reference Model SSL - Secure Sockets Layer TCP - Transmission Control Protocol TLS - Transport Layer Security UDP - User Datagram Protocol UTC - Universal Coordinated Time VXI - VME bus eXtension for Instruments Soloviev Expires Match 1, 2005 [Page 13]