LISTSERV historical documentation, release 1.5f ----------------------------------------------- Copyright Eric Thomas 1986,1987 +---------------------------------------------------------------+ | Revised LISTSERV: BITNET-Oriented Presentation | +---------------------------------------------------------------+ | | | Document number: P01-009-0 (February 28th, 1987) | | | | Author: Ross Patterson | | | | Document fileid: "LISTPRES MEMO" (from "Info PResentation") | +---------------------------------------------------------------+ Preface This manual is a general presentation of LISTSERV intended for BITNET, EARN and NetNorth node managers who have just installed a Revised LISTSERV or plan to do so in the near future. It can also be useful to other categories of users who seek a general yet technical intro- duction to LISTSERV and already have some experience in mailing lists and suchlike. It does not require any other particular knowledge from the reader. This document was written by Ross Patterson of Rutger's University for the Pre-SHARE presentation of LISTSERV that was held at SHARE 68 in San Francisco (February 28th, 1987). It has been found to be valuable enough to be included in the official LISTSERV library and distributed to all new LISTSERV nodes. You should note however that information contained in this manual will not necessarily be up to date at the time you read it. Please check the release number on the cover page and refer to the official Reference Guides (See "Appendix A. The LISTSERV Library") for more up-to-date information. ************************************ * General presentation of LISTSERV * ************************************ LISTSERV is a VM/CMS based Mailing Lists Manager, designed and written by Eric Thomas of the Ecole Centrale de Paris, France. It was designed to resolve several usability and control problems present in an earlier program, also called LISTSERV, written by Ricardo Hernandez for the BITNET Network Information Center (BITNET node BITNIC). The original BITNIC system has now been almost entirely replaced by the FRECP11, or "Revised", LISTSERV. The problems were well known, but bear repeating. From a usability standpoint, the single largest was that users could not join and leave lists on their own. The offical method was to send mail to the owner of the list, with a subject of "LISTSERV GROUPS", requesting to be added to or removed from the list. The owner's name and address was found by looking in a file called LISTSERV GROUPS, maintained by hand and available from the BITNIC file server, NICSERVE. Due to the constant but rapid rise in the number of lists, and the small amount of available BITNIC staff time, this list was often incorrect and always incomplete. In addition, requests, especially those to join the BITNIC based lists, often took weeks or months to satisfy. The Revised LISTSERV avoids these problems by allowing users to join and leave lists on demand, and by maintaining a list of available lists automatically. The next largest problem was that of network congestion. Mail from the list was sent separately to each subscriber. Some lists had upwards of 500 subscribers, with 10-20 submissions each day. The load caused by these lists was at times a serious problem. When the CUNYVM to BITNIC link was temporarily reduced to 19.2 Kbps in December 1985, the round trip time on a message was a horrendous 10 to 12 hours. The Revised LISTSERV avoids this problem by using a distributed model. A list can be housed on several LISTSERV nodes, called "peers". Each peer knows about the others, and they cooperate to the degree that there appears to be only one list. Users who subscribe to a list will have their request forwarded to the peer nearest to them, regardless of which peer received it. The set of peers can be, and usually is, different for each list. It is determined by the geographic spread in subscribers, as well as total readership and traffic volume. The actual choice of specific peers is left to the list owner, who must make arrangements with the maintainers of the other servers. A special list, REDIST-L@UGA, has been set up to discuss peering and to solicit new peers for lists. Since its inception, LISTSERV has had five releases, with numerous intermediate updates. It has grown from a basic Mailing List Manager to something much more. Along the way, it has gained such features as: o Efficient mass file distribution o Mail traffic archival o File server functions The network of peer-linked LISTSERVs has grown from a handful to 45 at last count, 33 of them on BITNET alone. The overall reduction in network load can only be guessed at, but it is believed by some to be as much as 75% on the larger lists. Recently, LISTSERV has passed several milestones. First, the BITNET Network Information Center, home to some of the largest lists on the network and origin of the LISTSERV concept, has replaced their older program with the new LISTSERV. While there have been some minor prob- lems, this replacement has been generally well received and quite successful. Second, the LISTSERV network has played an important role in the effort to reduce the bottleneck at the BITNET/ARPANet gateway, oper- ated by the University of Wisconsin. Over 50 ARPANet lists with large BITNET readership have had their BITNET subscribers moved onto LIST- SERV based lists, thus requiring that only one copy traverse the gateway. Third, the LISTSERV network has been used to distribute new releases of RELAY and Rice MAIL, as well as several electronic maga- zines. The design of LISTSERV stresses decentralization, in terms of both management and distribution. Users are grouped into three categories. o Postmasters are responsible for the installation, operation, and maintenance of one or more LISTSERV peers. o List Owners are responsible for the management of the individual mailing lists. o Subscribers receive mail and other files from the lists. There are also some commands for those who do not fall into any of these groups. List Owners usually have full control over their lists, on all peers that carry them. This control includes the ability to add and remove subscribers; maintain and collect usage statistics; suspend and resume distribution of mail and other files; and move subscribers from one peer to another. Postmasters have List Owner authority for all lists on their LISTSERV, although this privilege does not extend to other peers of these lists. In addition, they can stop and restart LISTSERV, maintain several critical control files, etc. Subscribers can query and change their distribution and acknowledge- ment options; change their names as recorded by LISTSERV; and, of course, leave the list. Those who are not subscribed to a list can get copies of the LISTSERV documentation; ask for help; get lists of available mailing lists; subscribe to lists; obtain copies of the list definition file, possibly including the subscribers list; obtain load statistics; get files from the File Server; and mass distribute files. Naturally, these privileges are also extended to the other user categories as well. LISTSERV is extremely flexible. Lists can be defined which are completely open, totally confidential, and just about anything in between. The List Owners specify what degree of confidentiality they require when coding the file that defines the list. Items under their control include: o The ability of users to join the list. o The availability of the subscribers list. o The availability of load statistics. o The ability to define who is authorized to contribute to the list. o Whether a list is to be included in the list of lists or treated as a Confidential list. o Archiving of message traffic. o List access by node or country. o The level of statistics to be gathered. o The default acknowledgement options. o and much more... LISTSERV accepts commands in three forms: As interactive messages; As mail; And as "Command Jobs" in any of several transmission formats. Interactive messages should be sent in the usual manner for your system: CMS: TELL LISTSERV AT (node) (command) TSO: TRANSMIT (node).LISTSERV '(command)' NOPROLOG VAX: SEND LISTSERV@(node) (command) Commands sent by messages will generally cause responses to be sent as messages, except where inappropriate. Commands can be enclosed in mail, where the mail text following the headers is treated as a file of commands. Each line is treated as a separate command, and the remainder of the file is flushed when an error occurs, such as an invalid command or bad syntax. Responses will normally be sent by mail back to the sender's mailbox. Command Jobs can either be a file of one line commands, processed exactly like the mail form, or a more flexible and of course more complex form. This form uses a control language that closely resem- bles MVS JCL, for ease of use by MVS users who are expected to need it most. LISTSERV offers two different distribution methods for files which are not formatted as mail. The first is the normal method, using a mailing list. Mail or files are sent to the list, where they are picked up and routed to all subscribers on the list, and all peer copies of the list. Mail is distributed via the Crosswell MAILER, according to the mailer informa- tion carried in the BITEARN NODES file. Separate copies are sent to each user on the LISTSERV node, and remote users are grouped by node. LISTSERV will combine copies for up to 5 subscribers on the same node when the receiving node has a MAILER to do the redistribution. Files which are not recognizable as mail are sent directly to the subscribers, bypassing the MAILERs. The second method is called Relayed File Distribution. Any user on any node can construct an RFD job using the Command Job Control Language, and send it to the nearest LISTSERV for processing. Each LISTSERV sends the files to those recipients nearest to it, and then forwards the remaining recipients to those LISTSERVs near it. This "peel-off" process insures that each recipient receives only one copy of the file, and that a distribution loop cannot occur. The job must list all the recipients by address, and can specify many control and monitoring options. The file to be distributed is enclosed in the job, and the whole job is sent as a single file to a nearby LISTSERV to begin its trek across the network. More information on Relayed File Distribution can be obtained from any LISTSERV by means of the INFO DIST command. The most recent addition to LISTSERV is the File Server. This facility allows authorized users to store a file on a disk under LISTSERV's control, specifying which users should be allowed to retrieve it. This is done by coding an entry in a FILELIST, which is simply a directory of logically related files. Many FILELISTs may exist, arranged in a tree structure. The root of the tree is LISTSERV FILELIST. This typically contains several other FILELISTs: CONTROL, which contains some critical system files; NOTEBOOK, which contains all of the archived mail traffic for lists that do so; and INFO, which contains all of the LISTSERV documentation. An entry in a FILELIST specifies the name of the file, its format and size, a brief comment, and the identities of those users who can store and retrieve, or PUT and GET, the file. These identities are recorded as File Access Codes, or FACs. A FAC is a three character abbreviation for an address or list of addresses, and must be unique within a FILELIST. Several special, or hard-coded, FACs have been defined for common cases: OWN, meaning the owner of a list; PRV, meaning Private, for the subscribers of a list; NAD, meaning Node ADministrators; ALL, meaning anyone; and N/A meaning that the process (GET or PUT) is not applicable to the file. Some of you may recognize this description as that of the EARN Network Server, NETSERV. The LISTSERV File Server was designed to resemble NETSERV as much as possible, so the commands and FILELISTs should be quite familiar to NETSERV initiates. Through the GET, AFD, FUI and INDEX commands, any user can obtain files stored by a LISTSERV. The INDEX command will cause a FILELIST to be sent back to the requestor, listing the available files and their access limitations. The GET command is used to request a file from a particular FILELIST. The AFD, or Automatic File Distribution, command requests LISTSERV to send a copy of a certain file each time it is changed. The FUI, or File Update Information, command is similar to AFD, but causes a short mail message to be sent noting that a new copy of the file is available. Finally, authorized users can store files via the PUT command, which is placed as the first line in the file to be stored. ************************************* * Appendix A. The LISTSERV Library * ************************************* o User's guide . . . . . . . . . . . . . . . . . . . . (U01-001) o List Manager's guide . . . . . . . . . . . . . . . . (M01-002) o Installation guide . . . . . . . . . . . . . . . . . (S01-003) o Application Programmer's guide . . . . . . . . . . . (A01-004) o Maintenance guide . . . . . . . . . . . . . . . . . . (S01-005) o File Server Functions . . . . . . . . . . . . . . . . (U01-006) o Listserv-Punch Implementation . . . . . . . . . . . . (R01-007) o File Maintainer's guide . . . . . . . . . . . . . . . (M01-008) --> o BITNET-Oriented Presentation . . . . . . . . . . . . (P01-009) LISTSERV Document Numbers ------------------------- U 01 - 006 - 0 _ __ ___ _ | | | | Document Class -----------+ | | | | | | | | | | | | Product Number --------------+ | | | | | | | | Publication Number -------------------+ | | | | Revision Number ------------------------+ Document Class The Document Class indicates for which category of persons the publi- cation was written. The current classes are: A Documents intended for Application Programmers. These publica- tions are usually very technical. M Documents intended for Software Managers, i.e. operators, "list owners", "file maintainers", et al. P General Presentation documents intended for persons who do not have any particular knowledge in the product. These are gener- ally non-technical documents. R Reference documents defining protocols used by the product. These documents are very technical and are intended for people who have to write interfaces for the product or attempt to port it to an operating system or environment for which it was not originally written. S Documents intended for Systems Programmers, i.e. the persons responsible for the installation and operation of the product. U Documents intended for General Users. Product Number The Product Number is a unique number associated with the product to which the publication relates. Number 01 refers to LISTSERV, number 02 corresponds to the NETINFO sub-product, etc. Publication Number This is a unique number associated with the publication. Publication Numbers are assigned sequentially, disregarding the Document Class. There is a different set of Publication Numbers for each product. Revision Number This number is incremented at every release change in the publication. Fractional numbers indicate intermediate changes between two releases.