NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
CAC Technical Memorandum No. 86
CCTC-WAD No. 7508
ARPANET RFC No. 725
NIC No. 38316
An RJE Protocol for a Resource Sharing Network
by
John Day
Gary R. Grossman
Prepared for the
Command and Control Technical Center
WWMCCS ADP Directorate
of the
Defense Communications Agency
Washington, D.C. 20305
under contract
DCAl00-76-C-0088
Center for Advanced Computation
University of Illinois at Urbana-Champaign
Urbana, Illinois 61801
March 1, 1977
Approved for Release - Peter A. Alsberg, Principal Investigator
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
For many users of the ARPANET, an RJE protocol is probably as important
in terms of utility as a TELNET (VTP) protocol. In fact, the facilities
provided by a TELNET and an RJE protocol are probably of most interest
to most users of computer networks. For these users, the net provides a
fast, cheap RJE surrogate, just as TELNET provides a telephone surrogate
for the timesharing user. The collection (and layers) of protocols that
provide these services must be organized to efficiently support a wide
variety of applications and user needs. They should not pose an undue
software burden on the user.
The "official" NETRJE protocol for the ARPANET has met an underwhelming
response from both the user and server community. I believe there are
two basic reasons. First, a large commitment of resources is necessary
to implement NETRJE. Second, the protocol creates serious security
problems.
In order to support the ARPA RJE protocol, a user must implement User
Telnet, Server FTP, and User RJE, while a server must implement Server
Telnet, User FTP, and Server RJE. In addition when an RJE session is
going on all three of these protocol implementations will be executing
for most of the life of the session. This could entail considerable
burden for some systems. Although it may not be out of line to require
a service to shoulder such burdens, it is out of line to require a user
to assume them in order to gain a rather basic service. Most user
installations are oriented toward meeting their user's needs not toward
implementing large amounts of network software. (In fact one of the
better aspects of the previous ARPANET protocol designs was that they
attempted to minimize the work for the user. (It must be admitted
though that compassion for the user was not the reason for this
approach.)
In order to support a "hot line printer" (i.e., a job is automatically
printed when it is completed), the user must store his user code and
password for the output host at the server host. This, of course,
presents a rather severe security problem. Although the ARPANET can not
be made totally secure without massive revision, there are some basic
precautions that can be taken to protect users from being victimized by
every first year Computer Science student with access to the net.
The RJE protocol proposed here tries to mitigate the implementation
problems and security problems. The protocol is designed to provide
three levels of service. A user or server has the perogative to
implement the protocol at whatever level their resources allow. The
service can then be upgraded to cleaner or more sophisticated approaches
when and if the opportunity arises.
This protocol is described in terms of the ARPANET. Several aspects of
[page 1]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
the design (such as the reply structure) were made to coincide with
existing ARPANET conventions. This was done to facilitate understanding
and limit the discussion to the protocol itself. Although the protocol
is described in ARPANET terms, it should be applicable to other network
environments.
This paper is not considered to be complete in every detail. It was
written primarily to elicit comments from the network community and to
measure the desire of the community to adopt such a procedure. We have
tried to describe enough of the protocol so that the reader can get an
idea of how things are to work without getting bogged down in the detail
that would be necessary for implementation. Below is an outline of the
final protocol document as presently conceived. Sections marked with an
asterisk are to be provided later.
[page 2]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
Introduction
Part I
The NETRJE Models
1. Telnet (VTP) Model
2. Telnet with Data Transfer Model
3. Telnet with FTP Model
Scenarios for the Models
* Suggested Implementaton Schemes for Various Applications
Part II
The Server RJE Commands
* General Conventions
Commands
Replies
Numerical List
Command-Reply List
* Details of the Data Transfer
* Minimal Requirements for a User RJE
* Minimal Requirements for a Server RJE
* Glossary of Terms
[page 3]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
Part I THE NETRJE MODELS
------------------------
This section describes the proposed NETRJE protocol in a narrative form.
A formal definition will be included in Part II after review. The
narrative should provide the general reader with the flavor of the
protocol without getting bogged down in unnecessary detail. The
proposed NETRJE protocol provides three different models for job
submission and retrieval. The three models can be characterized as 1)
RJE using Telnet only, 2) RJE using Telnet and Data Transfer, and 3) RJE
using FTP. This approach provides flexibility for both implementors and
users. User and server sites constrained by manpower or machine
resources may implement only the simpler models. The user may use the
different models separately or in any consistent combination which best
suits his requirements and convenience. Servers should assume that the
minimal implementation of a more sophisticated model includes the
minimal implementations of all less sophisticated models. (There are,
however, certain minimal requirements that must be supported.) This
secton will discuss each of these models in turn, and show each one can
be used to provide a useful network RJE functon.
This protocol does not contain the security difficulties of the present
protocol. This has been avoided by requiring that the burden of
implementing the "hot line printer" or "hot card reader" be put on the
user system. Thus, those systems which desire such a facility may still
support it. The user implementaton will be slightly more complicated.
The trade-off is the increased security of the protocol.
End-to-end protocols are assumed to be available and to provide an
ordered, error free bit stream to the RJE protocol. It is also assumed
that a suitable virtual terminal protocol such as Telnet, is used to
format the control connection.
RJE Using Only Telnet (VTP)
---------------------------
The intent of this model is, bluntly, to provide an official "quick and
dirty" form of the protocol. Many organizatons, both users and servers,
are often confronted with problem of providing a service quickly or
within very tight budgetary constraints. This model is intended for
these situations. With this model, the user is required only to be able
to establish a Telnet connection via the RJE contact socket. Commands,
replies, and data are all sent over the Telnet connecton. Card input or
printer output has the appearance of coming from or going to the user's
terminal. The user's system may allow output to be diverted from the
terminal to another device such as the line printer. The technique of
diverting terminal output was used with great success in the MARK I ANTS
[page 4]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
systems where various devices were not assigned socket numbers as in a
TIP. This technique is also useful for hosts that allow program access
to the network only through the user's Telnet connection. This
situation may exist in the early phases of a server's availability to
the network. When data is transferred in this mode an end-of-data
marker will be sent to aid the receiving host in determining when to
stop diverting the data. This model will have to handle the problems of
data traveling on a connection essentially meant for control. The use
of this data transfer mechanism is intended as an intermediate measure
required by limited resources. For now we let it stand that the
designers are aware of the problems inherent in embedding commands or
replies in the data stream. We will leave the exact resolution of the
problem to the formal definition.
This proposed NETRJE protocol uses a schedule verb, SCHED for job
submission. For this model, there are two forms of SCHED that are
relevant. First, there is the "SCHED " form. This
command indicates to the server that there exists at the server site a
file with all necessary job control information and data to define a
job. The server will then attempt to place the job in the job queue and
reply to the user indicating success or failure and possibly a job-id.
This job-id will be used when inquiring about the job status or
retrieving the job's output.
When the job finishes, the server will take one of two actions:
a) if the user is still logged in, the server will send a reply
notifying the user of his job completion; or,
b) if the user is not logged in, the server will save the status of
the job which may later be interrogated via the STATUS command (see
below).
The otherform of SCHED of relevance to this model has the syntax:
SCHED INPUT .
This allows the user to sit down at a terminal and type his own job
control or possibly a program. It also allows those users whose local
systems provide a facility to transmit files with User TELNET to
transmit user input job fles in this way. The RJE Server would insert
the job into the local job stream, returning the proper indication of
success or failure along with identification of the job.
Just as the SCHED command provides several ways for job submission, the
OUTPUT command provides several options for retrieving output. The form
[page 5]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
OUTPUTDISCARD
is sent to the server to initiate the output to the user's site
according to output specifications defined by previous OUTDEF commands
(see below). The optional DISCARD argument to the OUTPUT command
indicates, if present, that the file is to be destroyed after
transmission has completed successfully.
The OUTDEF command for a job may be sent at any time after the job has
been scheduled and before it is retrieved using the OUTPUT command.
This command will specify the parameters necessary to effect the
transfer of the output to the user or to define the disposition of the
output. We realize that the OUTDEF command
(indicating that output is to be placed in a file described by the
pathname) may be difficult for some systems to implement. These systems
would merely respond negatively indicating their inability to perform
the function.
A scenario is now in order to illustrate the model. The user has logged
in to Multics and is ready to submit an RJE job in the following way
(XXX will denote the as yet unspecified reply code for the reply):
SCHED MY-JOB>TREK
The system responds with a reply indicating the job has been submitted
successfully and returns a job-id, say XA1423.
XXX JOB XA1423 was successfully submitted.
At some later time a message appears.
XXX JOB XA1423 has completed.
The user or user process now sends OUTDEF XA1423 TELNET indicating that
the job should be sent on the TELNET connection. A reply returns
XXX last command successful.
The user now sends
OUTPUT XA1423
and the server replies with
XXX Output ready. Type an empty line when ready.
The user then sends an empty line when he is read to receive the output.
[page 6]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
This exchange allows the user to effect output diversion at his local
site if necessary after he has confirmed the server is ready.
If the user had not wished to wait on his output and had logged off
after getting the successful submission, the next time the user logged
in he could inquire as to the status of the job or all jobs under his
usercode and then proceeded to output any or all of them.
RJE with TELNET and Data Transfer
---------------------------------
The previous model provided a minimal implementation for NETRJE. This
model provides better data transfer facilities without requiring an FTP
implementation. This model requires no new commands, but does
manipulate connections differently, so that data is not required to flow
on the command connection (see Fig. 2). Data is sent on separate
default connections (unless otherwise specified) as in the CCN NETRJS
protocol. However, for this protocol the defaults used will be the same
offsets from the control connection as those in FTP.
The use of this model is indicated to the Server by either the INDEF
command or a SCHED command with no arguments. The INDEF command allows
the user to specify a socket other than the default socket as the source
of the input. On receipt of the SCHED or INDEF indicating this
technique is to be used, the Server will attempt to connect to the
appropriate socket. If a SCHED command was sent, the user protocol
interpreter could start sending cards as soon as the data connection is
established. (It is assumed that the user interface has indicated to
the RJE protocol interpreter where the cards are to come from.) If the
command was INDEF, then the Server will not start reading until the
SCHED is received. Similarly, when the output is ready, either an
OUTDEF or OUTPUT command is sent to set up and start the printing. The
INDEF and OUTDEF commands used with this mode will also allow moving
data to or from a TIP or printer.
This model requires definiton of actual data transfer formats for the
reader and printer lines. We propose that the formats and connection
schemes of the present FTP be adopted. This solution has the advantage
of not requiring extra coding efforts for users with FTP implementations
and may allow them to organize their FTP implementations and may allow
them to organize their FTP and NETRJE implementations in such a way as
to take advantage of common algorithms. One might easily confuse this
solution with a revival of the Data Transfer Protocol. Some thought on
a more rigorous definition of a Data Transfer Protocol for the common
use of FTP and RJE might be worthwhile in the future.
[page 7]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
Let us consider a scenario.
The user wishes to submit a card deck to the Server. He then types
SCHED
The Server opens a connection to the user's default card reader socket
while sending a reply to the user on the control connection.
XXX attempting connection to card reader.
When the connection is opened, another reply:
XXX transfer started
and when completed:
XXX JOB XA 1423 was successfully submitted.
When the job completes and the completion message is sent to the user,
he may wish to send the output to his TIP printer on socket Y. He will
then type
OUTDEF XA1423 255, Y (255 being his host address).
The Server will then attempt to connect to the socket and will reply
XXX printer connection successful.
When the user has satisfied himself all is in readiness, he will type
OUTPUT XA1423
and the Server will start sending and reply to the user
XXX print started.
When the transfer is complete the Server will close the data connection
and send an appropriate reply.
[page 8]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
NETRJE Using FTP
----------------
This model (illustrated in Fig. 3) uses FTP to effect the transfer of
the files. It may be easier for some systems to use this sort of model
for more sophisticated RJE systems. This is especially true if the
users desire to take input from the local file system or to send output
to the local file system rather than from an actual card reader or to an
actual line printer. Although using the local file system is not
prohibited by the Data Transfer model, it may be easier to approach
through FTP. Using FTP with NETRJE also allows the utilization of the
FTP server-server transfer mechanism to generate input from or direct
output to a third host.
The only new facility required by this model are the commands INPATH and
OUTPATH. When using FTP to transfer input to the Server, the user must
know where to send the job so that it enters the job stream. The INPATH
command returns as a reply such a legal pathname. Thus the scenario for
job submission is as follows: The user sends an INPATH command; the
Server responds with a legal Server pathname for the user. The user
process starts sending the input to the file using FTP. When transfer
is complete, the user sends a SCHED command. When the
job has finished, the pathname created for the user may or may not
destroy the input file. The OUTPATH command is similarily used to
identify the pathname for the output, so that it may be retrieved by
FTP. Some systems may define file names in such a way that the user may
derive them from the parameters of his job.
Note on Replies
In all of the above examples we have refrained from defining actual
reply codes. The intent is to use the same reply structure, and where
appropriate the same numbers, as described in RFC 640 "Revised FTP Reply
Codes".
Protocol Measurement
An integral part of any good protocol definition is a set of
measurements to allow evaluation of both the protocol and its
implementation. This provides two functions: 1) It allows the protocol
designer to evaluate the protocol and make improvements. 2) It allows
the user of the protocol to know how expensive it is and to demand
improvements. The proposed NETRJE protocol provides two sets of
measures - one for a particular session and one for overall performance.
These measurements may be elicited by the MEASURE command which will
take an argument with three values: JOB (job statistics and cost
measurements), SESSION (measurements taken for this sesson), and GLOBAL
[page 9]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
(overall measurements of the performance of the protocol and its
implementation). The command will return the measurements in a fixed
format reply.
The measurements reported for a job are:
1. CPU time,
2. I/O operations,
3. storage space time product,
4. job cost in dollars,
5. elapsed time the job waited before being executed, and
6. elapsed time for the job to execute.
The measures taken from a sesson are:
1. number of bits transferred,
2. transmission rate of input or output transfers,
3. the amount of CPU time, storage space-time product, and I/O
operations for the session.
4. cost in dollars and cents.
The measures to be taken globally are:
1. frequency of commands and possibly command forms,
2. model frequency (which submission/retrieval model used),
3. transmission mode frequency,
4. total number of sessions,
5. transmission rate: average, std. deviation, upper and lower
bounds (also by transmission mode),
6. cpu time, storage space-time product, and I/O operations for both
the protocol and jobs submitted: average, std. deviation, and upper
and lower bounds (overall as well as by model, transfer mode, and
file size). (The reason for including job statistics here is so that
[page 10]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
management and systems personnel have some indication how the
facility is being used.)
It is clear that it may be difficult to acquire some measures (such as
transmission rate) when NETRJE is using FTP. This is unavoidable since
FTP is not metered. The most straightforward solution is also to meter
FTP (hint). For the final definition a close look will be given to the
subset that should be required. Comments are welcome. However, we
believe strongly that it is very important to know how a facility like
this is used as well as how well it performs.
[page 11]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
Part II. Preliminary Definition of NETRJE Commands
---------------------------------------------------
For purposes of discussion this section gives a very preliminary
definition of the NETRJE commands and their replies. The intent is to
give a brief but not exhaustive definition of each command and its major
replies to give the flavor of the protocol. We do not do this to
discourage nit-picking by critics, since we may actually overlook the
obvious on occasion, but merely to expedite the writing of this paper.
The reply scheme will follow the model of the revised FTP reply codes
described in RFC 640.
Access Control
USER
PASS
ACCT
These perform the normal functions to log the user into the system. The
replies to them are the standard ones in FTP. It was never clear why
"account" was not included in the old NETRJE. Presumably, if it's
necessary for an FTP or Telnet user, it will be necessary for an RJE
user.
REINIT
This command reinitializes the state of the NETRJE server process so
that it is ready for a new user. If the transfer of data is in progress
for the previous user, it will be allowed to complete.
ABORT
This command is used to abort the transfer of data. This command is
meaningful to the Server only if the data is being transferred over the
Telnet connection or the default data sockets. If FTP is being used,
the execution of this command is the responsibility of the USER NETRJE
process.
BYE
This command causes the Server to log out the user and close the Telnet
connection. If the transfer of data is in progress, the action of the
command will be delayed until the transfer is complete.
[page 12]
NWG/RFC# 725 DAY GRG 25-APR-77 12:41 38316
An RJE Protocol for a Resource Sharing Network
SCHED
RFC, FYI, BCP