INTERNET-DRAFT | F. (Lalo) Martins July 1996 | Independent software developer Expires: 4/dec/96 | Linux Objex development team Internet Discussion Forum Protocol (IDFP) draft-martins-forums-00.txt Status of this Document This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Abstract This document presents a new alternative way of implementing discussion forums other than NNTP and mailing lists. It is largely based on standard e-mail except for retrieval. The purpose of this new protocol is to atenuate the bandwidth and net resources needed by either NNTP and mailing lists. Introduction With the growth of the Internet, its two means of discussion (one-to-many) communications have grown with it reaching levels such that they can safely be assumed to represent a good portion of all network traffic. When these protocols were created, yers ago, there weren't enought users to make bandwidth an issue; instead people wanted a distributed model (NNTP) to allow for low speeds. Now, holding thousands of copies of the newsgroups in servers all over the world has proven to be a waste of resources, since we could easily access them on one, or at most, for the top groups, ten hosts through the world. The need for the newsgroups to be copied by this much hosts has made the last years see an unthinkable explosion of mailing lists. As they are a good substitute for newsgroups as long as postings don't have to be stored in lots of places, they are inherently closed (if Martins [Page 1] Draft Internet Forums July 96 someone is interested on them, it is necessary to subscribe) and aren't by definition stored anywhere - archives have to be made "by hand" by someone, usually the list mantainer. Also, mailing lists waste a lot of bandwidth since subscribers don't always read all postings - unread postings have to be transmitted via SMTP, usually trough many hosts, as well as those wich are actually read. This documment presents a mid-term between newsgroups and mailing lists, called Internet Forums. These are treated like mailing lists half of their lives; someone posts a standard mail message to them just like posting to a mailing list, and messages are carried to the Forum host just like any mail message. Once they are in the Forum host, they become similar to newsgroups in that they're not redistributed, but instead kept and archived for later reading by anyone through a retrieval protocol similar to POP. Table of Contents This document is presented in five chapters and two appendices. 1.Description of the protocol ..................................... 3 (brief discussion about this all) 2.Message retrieval ............................................... 4 (description of the core of IDFP: the retrieval subsystem) 2.1.Starting the session .......................................4 2.2.Listing messages ...........................................5 2.3.Retrieving messages ........................................6 2.4.Ending the session .........................................6 2.5.Example session ............................................6 3.Moderating forums ............................................... 8 (instructions on how to implement moderated forums) 4.Mirroring ....................................................... 9 (achieving some of NNTP functionality trough mirroring) 5.Security considerations ........................................ 10 (what NOT to expect from IDFP) a.References ..................................................... 10 b.Author's Address ............................................... 10 Just ten pages... wish more internet standards were like this! Martins [Page 2] Draft Internet Forums July 96 1.Description of the protocol The Internet Discussion Forum Protocol presents methods for accessing, listing and reading Internet Discussion Forums, and to implement moderated and mirrored Forums. The method for submitting messages is SMTP (Simple Mail Transfer Protocol) and is beyond the scope of this document - refer to RFC 821. The method for storage is up to the implementor and is beyond the scope of this document. The format of the messages is MIME and is beyond the scope of this document - refer to RFC 1522. IDFP is intended to be: 1: simple, so that hackers, people who are experimenting with the system or otherwise brave people can access it directly with any telnet tool; 2: similar to POP (Post Office Protocol), so that POP programmers can easily start doing Forum clients; 3: not too space-consumming so that it fulfills its primary goal of freeing bandwidth and so that people can easily read it over slow modem connections. Forums have an standard "e-mail" address trough wich they receive they messages. As any a-mail address, it is composed of the Forum name (internal host identification), an at "@" sign and the host name. For example, if we would copy the newsgroup comp.os.linux.misc to a forum at, say, forums.linux.org (wich would be a bad idea BTW), its address would be "comp.os.linux.misc@forums.linux.org". Note that this address is somewhat big; forum addresses are restricted in size just like any address. Martins [Page 3] Draft Internet Forums July 96 2.Message Retrieval The method for retrieving Internet Forum messages is designed to be simple and similar to POP. Its description is divided in four stages, with the second and third being optional (you can connect, list and close, connect, retrieve and close or even, for any reason, connect and close). There is just one command for each stage, and commands are not case-sensitive (retr is the same as RETR or Retr). IDFP uses the response model from POP, issuing just one "ok" and one "err" message. This is intended for simplicity, a goal POP successfully reached. But while POP issues "+OK"and "-ERR" messages, wich must always be uppercase, IDFP messages are always lowercase "+ok" and "-err" to add a layer of differentiation - thus on connection any user agent can tell if the host is POP or IDFP just by looking at the welcome message. IDFP also adds a "-cri" response to notify critical errors. A critical error is defined as any error wich causes closing of the session by the server. Also, there are no provisions for listing the forums on a server. To read a forum, the user has to already have its address. 2.1.Starting the session The session starts when the computer interested on receiving the messages (from now on referred to as the client) connects via TCP to the computer that keeps the messages (from now on referred to as the host or server) trough a well-nown port on wich the host was left "listening". As long as the connection is stablished the host should respond with a line starting with "+ok". This message shouldn't exceed one line. The client is expected to reply with the keyword "user" and the full e-mail address of the user. This is for log purposes only. It is also possible that the host use this information to restrict multiple connections by the same user, but this is an optional behaviour. Once the user is identifyed, the server should again respond with "+ok" and start the session, entering a state in wich it is ready for the IDFP commands "list", "retr" and "quit". If there is an error anytime before identification, such as the user being already connected (and multiple sessions not being permitted), the user not being welcome (for reasons up to the host's mantainer) or any internal server error, the server should respond with "-cri" and close the connection. Martins [Page 4] Draft Internet Forums July 96 2.2.Listing messages The command for listing messages is "list". There are two parameters, of wich the first is required and the second is optional. The first parameter is the name of the forum, without the "@" and the hostname. The second is the number of the first message to list. Each message in a forum must have an unique number. If old messages are deleted, newer ones should retain their number and further posts shouldn't reuse numbers. So if an user agent has already listed up to "500", it is safe to assume that any new messages can be listed with "list forumname 501". The server should respond with, first, "+ok", then a blank line, then the listing. Entries in the listing are separated by a blank line. The listing ends on a line composed of a period "." by itself. Each entry in the listing is composed of part of the MIME headers for the respective headers. The first line is the internal message number. Note this line isn't part of or incorporated in the header. The format is: "Message number: xxxxxx", according to RFC 822 format for headers. Then follows the "Subject", "From", and "Date" message headers. Example listing: ------------------------------------ list foobar 237 +ok last message in foobar is 240 Message number: 237 Subject: Best foobar on the net? From: john.doe@mib.mil Date: Sun, 28 Jul 1996 17:18:19 -0700 Message number: 238 Subject: Re: fresh fish from foo freezer From: jose.da.silva@provedor.com.br Date: Sun, 28 Jul 1996 18:00:25 -0300 Message number: 239 Subject: Re: Best foobar on the net? From: dante@crystal.net Date: Sun, 28 Jul 1996 19:33:06 -0600 Message number: 240 Subject: Re: Best foobar on the net? From: john.doe@mib.mil Date: Sun, 28 Jul 1996 19:50:13 -0700 . Martins [Page 5] Draft Internet Forums July 96 2.3.Retrieving messages The retrieval command is "retr". Its two paramenters are the forum name and the message number, both required. The server response is similar to the POP response for the POP RETR comand, except the "+OK" message should be replaced by "+ok". The host first issues a "+ok" message; then a blank line; then the message, in standard MIME format, ending with a line with a period "." by itself. As in POP, lines of the actual message wich start with a period should be preceded with another period. 2.4.Ending the session The session is closed when the client issues the "quit" command. The server is only permitted to close the session after receiving a "quit" command and issuing a "+ok" response, or if any internal error occurs and after issuing a "-cri" message. 2.5.Example session The lines starting with "%" are entered by the client; please note there is not an actual "%" in the session. Note also that sending blank lines after responses is an optional but recommended behaviour of the server. Martins [Page 6] Draft Internet Forums July 96 +ok Foonet Internet Forum server ready at foonet.org %user lalo@webcom.com +ok be welcome %list foobar 240 -err no new messages %list enfoozer.devel 79 +ok 3 messages (1522 octets) Message number: 79 Subject: Re: fresh fish from foo freezer From: jose.da.silva@provedor.com.br Date: Sun, 28 Jul 1996 18:00:25 -0300 Message number: 80 Subject: Re: fresh fish from foo freezer From: dante@crystal.net Date: Sun, 28 Jul 1996 22:16:00 -0600 Message number: 81 Subject: Should we add a deffozer? From: john.doe@mib.mil Date: Mon, 29 Jul 1996 02:00:22 -0700 . %retr 81 +ok 300 octets Received: from mib.mil by foonet Mon, 29 Jul 1996 02:05:08 -0700 Received: from localhost by mib.mil with SMTP (1.37.109.15/16.2) \ id AA234939381; Mon, 29 Jul 1996 02:00:30 -0700 Date: Mon, 29 Jul 1996 02:00:22 -0700 Message-Id: <31F8FFE6.8BC@mib.mil> From: John Doe Organization: Men In Black Mime-Version: 1.0 To: enfoozer.devel@foonet.org Subject: Should we add a deffozer? Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Content-Length: 300 I think a good enffozer tool also has to have a deffozer included. Comments? John Doe - Man in Black (john.doe@mib.mil, http://www.mib.mil/doe) . %quit +ok foonet.org forums server closing connection Martins [Page 7] Draft Internet Forums July 96 3.Moderating forums Sometimes there is need to disable postings to a forum, except from one person (usually tha mantainer) called the Moderator. To do so, once a message for the forum arrives at the host it is forwarded to the moderator, instead of added to the forum. This part is perfectly possible trough current standards. What this document has to propose is a method for the moderator to actually submit the approved messages. Of course the easiest way of implementing such a feature is locally by someone working on the same LAN as the server, trough direct editing of the forum files. While this is a permitted way of doing things, we also must make provisions for moderators who don't work at and don't have access to the server. Thus, the recommended behaviour for servers wich receive messages for a forum marked as "moderated" is to check if it is from the moderator; if it is, add it to the forum; if not, forward it to the moderator. Messages from the moderator aren't identified by the "from" header but instead by a special "subject" header. Instead of the default subject, this should contain the keyword "Moderator", followed by a colon ":", a space, and a cryptic host-specific key. While this key can't be internationally stardarized since this would go against its very purpose, it is recommended that it is based on the message body (something like a checksum) plus a secret moderator password. The message itself shouldn't contain anything; instead it should have, in the form of MIME "attachments", the actual message(s) the moderator is submitting. Note that since we're using MIME it is possible to submit many messages in one post. It is also possible for a forum to be "not postable", meaning ONLY THE MODERATOR is able to post. Valid examples are the "*.announce" newsgroups where people are not expected to post, just read. The way this behaviour is implemented is up to the programmer - just like a forum can be marked as "moderated" it can also be marked "not postable" in wich case all postings to it are either discarded or returned. Accepting postings from the moderator works the same way. Note that all the behaviours described here are optional. Any particular server program can choose not to implement moderated forums at all, or if it does implement it it may not implement the validated-posting from the moderator, if it is going to be used in an environment where moderators will all work in the internal network, and even then be considered compliant with this document. What programs are required to do is to implement things as a whole. If moderated forums are implemented, "not postable" ones must be too. If there is a provision for remote moderators at all, it must be the one described in this section. Martins [Page 8] Draft Internet Forums July 96 4.Mirroring Very popular forums can result in cluttered hosts, and the consequence is a slow or even refused connection, especially if the client is located far from the host. Fortunately, as this also happens to the web and to ftp sites, the solution has already been thinked out for us. It is called Mirroring, and consists on copying a site - or, in this case, forum. Different from Web and ftp mirrors, Forum mirroring is done mostly by the main host. It should keep a list of mirror sites and forward new messages to all the mirros. A good way of doing this, if there is more than one mirror, is to put them in a mailing list. In all other aspects mirrors should act just like moderated lists, and the main host should accordingly behave like a moderator. So messages are sent from the main site to the mirrors encapsulated in a MIME "envelope" with a "passcode" subject field. It is also possible to send messages to the mirror forums. These messages are just forwarded to the main site. Remember in all aspects a mirror list is a moderated list with the main host being the moderator. Martins [Page 9] Draft Internet Forums July 96 5.Security considerations Inherently, Internet Forums are not secure. Anything even remotely private shouldn't be on a public forum. The only security issues addressed by this document are concerning authentication of moderator messages, in section 3. A certain degree of privacity can be achieved using standard tools for electonic mail encription, if something must be read by a restricted but large group and nobody else. Since Forum messages are MIME messages, almost everything that applies to electronic mail security applies also to Forums. Appendix A.References The RFCs and STDs on SMTP, POP and MIME, and RFC 822. Ok this isn't how a "references" section is done, but then this is a draft and I promise to have one in the final version of this. Appendix B.Author's Address Fernando Mauro Martins - aka Lalo postal: Rua Realengo, 268 - cep 05451-030 - Sao Paulo - Brazil phone: +55-11-263 7358 (home and operating base) now how to REALLY contact me: http://webcom.com/lalo | mailto:lalo@webcom.com Martins [Page 10]