Copyright © 1994 Free Software Foundation, Inc.

Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions.

Introduction
************

This manual describes the interfaces that make up the GNU Hurd.  It is assumed that the reader is familiar with the features of the Mach kernel, and with using the Hurd interfaces as a user, and all of the associated C library calls.  It concentrates on requirements and advice for the writing of Hurd servers, as well as describing the libraries that come with the GNU Hurd.

It is assumed that the reader of the manual is perusing the referenced MiG interface definitions and library header files for the section being examined.

I/O interface
*************

The I/O interface is used to interact with almost all servers in the GNU Hurd.  It provides facilities for reading and writing I/O streams.  The I/O interface facilities are described in <hurd/io.defs> and <hurd/shared.h> The latter portion of <hurd/io.defs> and all of <hurd/shared.h> describe how to implement shared-memory I/O operations, and are described later.  The present chapter discusses RPC-based I/O operations.

I/O object ports
================

The I/O server must associate each I/O port with a particular set of uids and gids, identifying the user who is responsible for operations on the port.  Every port to an I/O server should also support either the file protocol or the socket protocol; naked I/O ports are not allowed.

In addition, the server associates with each port a default file pointer, a set of open mode bits, a pid (called the ``owner''), and some underlying object which can absorb data (for write) or provide data (for read).

The uid and gid sets associated with a port may not be visibly shared with other ports, nor may they ever change.  The server must fix the identification of a set of uids and gids with a particular port at the moment of the port's creation.  The other characteristics of an I/O port may be shared with other users.  The I/O server interface does not generally specify in what way servers may share these other characteristics are shared (with the exception of the deprecated O_ASYNC interface); however, the file and socket interfaces make further requirements about what sharing is expected and prohibited from occurring.

In general, users get send-rights to I/O ports by some mechanism that is external to the I/O protocol.  (For example file servers give out I/O ports in response to the dir_pathtrans and fsys_getroot calls.  Socket servers give out ports in response to the socket_create and socket_accept calls.)  However, the I/O protocol provides methods of obtaining new ports that refer to the same underlying object as another port.  In response to all of these calls, all underlying state (including, but not limited to, the default file pointer, open mode bits, and underlying object) must be shared between the old and new ports.  In the following descriptions of these calls, the term ``identical'' means this kind of sharing.  All these calls must return send-rights to a newly-constructed Mach port.

The io_duplicate call simply returns another port which is identical to an existing port and has the same uid and gid set.

The io_restrict_auth call returns another port, identical to the provided port, but which has a smaller associated uid and gid set.  The uid and gid sets of the new port are the intersection of the set on the existing port and the lists of uids and gids provided in the call.

Users use the io_reauthenticate call when they wish to have an entirely new set of uids or gids associated with a port.  In response to the io_reauthenticate call, the server must create a new port, and then make the call auth_server_authenticate to the auth server.  The rendezvous port for the auth_server_authenticate call is the I/O port to which was made the io_reauthenticate call.  The server provides rend_int parameter to the auth server as a copy from the corresponding parameter in the io_reauthenticate call.  The I/O server also gives the auth server a new port; this must be a newly created port identical to the old port.  The auth server will return the set of uids and gids associated with the user, and guarantees that the new port will go directly to the user that possessed the associated authentication port.  The server then identifies the new port given out with the specified id's.

Simple operations
=================

Users write to I/O ports by calling the io_write RPC.  They specify an offset parameter; if the object supports writing at arbitrary offsets, the server should honor this parameter.  If -1 is passed as the offset, then the server should use the default file pointer.  The server should return the amount of data which was successfully written.  If the operation was interrupted after some but not all of the data was written, then it is considered to have succeeded and the server should return the amount written.  If the port is not an I/O port at all, the server should reply with the error EOPNOTSUPP.  If the port is an I/O port, but does not happen to support writing, then the correct error is EBADF.

Users read from I/O ports by calling the io_read RPC.  The specify the amount of data they wish to read and the offset.  The offset has the same meaning as for io_write above.  The server should return the data read.  If the call is interrupted after same data has been read (and the operation is not idempotent) then the server should return the amount read, even if less than the amount requested.  The server should return as much data as possible, but never more than requested by the user.  If there is no data, but there might be later, the call should block until data becomes available.  Indicate end-of-file conditions by returning zero bytes.  If the call is interrupted after some data has been read, but the call is idempotent, then the server may return EINTR rather than actually filling the buffer (taking care that any modifications of the default file pointer have been reversed).  Preferably, however, servers should return data if possible.

There are two categories of objects: seekable and non-seekable. Seekable objects must accept arbitrary offset parameters in the io_read and io_write calls, and to implement the io_seek call.  Nonseekable objects must ignore the offset parameters to io_read and io_write, and should return ESPIPE to the io_seek call.

On seekable objects, io_seek changes the default file pointer for reads and writes.  (See the C library manual for the interpretation of the WHENCE and OFFSET arguments.)  It returns the new offset as modified by io_seek.

The io_readable interface returns the amount of data which can be immediately read.  For the special technical meaning of ``immediately'', see Asynchronous I/O.

Open modes
==========

The server associates each port with a set of bits that affect its operation.  The io_set_all_openmodes call modifies these bits and the io_get_openmodes call returns them.  In addition, the io_set_some_openmodes and io_clear_some_openmodes do an atomic read/modify/write of the openmodes.

The O_APPEND bit, when set, changes the behavior of io_write when it uses the default file pointer on seekable objects.  When io_write is done on a port with the O_APPEND bit set, is must set the filepointer to one more than the ``maximum correct value'' (described below) before doing the write (which would then increment the file pointer as usual).  The server must atomically bind this update to the actual data write with respect to other users of io_read, io_write, and io_seek.

A ``correct value'' for the file pointer which, when provided to io_read, will successfully return at least one byte of data and not end-of-file. The ``maximum correct value'' referred to in the description of O_APPEND is the maximum such correct value.  (For ordinary files [see the description of the file protocol for more information] this is the same as the current file size.)

The O_FSYNC bit, when set, causes io_write not to delay writing data to underlying media in any fashion.

The O_NONBLOCK bit, when set, prevents read and write from blocking. They should copy such data as is immediately available.  If no data is immediately available they should return EWOULDBLOCK.

The definition of ``immediate'' is more or less server dependent.  Some servers (disk-based file servers, most notably) regard all data as immediatebly available.  The one criterion is that something which must happen immediately may not wait for any user-synchronizable event.

The O_ASYNC bit is deprecated; its use is documented in the following section.  This bit must be shared between all users of the same underlying object.

Asynchronous I/O
================

Users may wish to be notified when I/O can be done without blocking; they use the io_async call to indicate this to the server.  In the io_async call the user provides a port on which will the server should send sig_post messages as I/O becomes possible.  The server must return a port which will be the reference port in the sig_post messages.  Each io_async call should generate a new reference port.  (See the C library manual for information on how to send sig_post messages.)

The server then sends one SIGIO signal to each registered async user everytime I/O becomes possible.  I/O is possible if at least one byte can be read or written immediately.  (The definition of ``immediately'' must be the same as for the implementation of the O_NONBLOCK flag.)  In addition, everytime a user calls io_read or io_write on a non-seekable object, or at the default file pointer on a seekable object, another signal should be sent to each user if I/O is still possible.

Some objects may also define ``urgent'' conditions.  Such servers should send the SIGURG signal to each registered async user anytime an urgent condition appears.  After any RPC that has the possibility of clearing the urgent condition, the server should again send the signal to all registered users if the urgent condition is still present.

A more fine-grained mechanism for doing async I/O is the io_select call. The user specifies the kind of access desired, and a send-once right. If I/O of the kind the user desires is immediately possible, then the server should return so indicating, and destroy the send-once right.  If I/O is not immediately possible, the server should save the send-once right, and send a select_done message as soon as I/O becomes immediately possible.  (Again, the definition of ``immediate'' must be the same for io_select, io_async, and O_NONBLOCK.)

For compatibility, the I/O interface provides a deprecated feature (known as icky async I/O)..  The calls io_mod_owner and io_get_owner set the ``owner'' of the object, providing either a pid or a pgrp (if the value isnegative).  Whenever the I/O server is sending sig_post messages to all the io_async users, if the O_ASYNC bit is set, the server should also send a signal to the owning pid/pgrp.  The ID port for this call should be different from all the io_async id ports given to users. Users may find out what ID port the server uses for this by calling io_get_icky_async_id.

Information queries
===================

Users may call io_stat to find out information about the I/O object. Most of the fieds of a struct stat are meaningful only for files.  All objects, however, must support the fields st_fstype, st_fsid, st_ino, st_atime, st_atime_usec, st_mtime_user, st_ctime, st_ctime_usec, and st_blksize.

st_fstype, st_fsid, and st_ino must be unique for the underlying object across the entire system.

st_atime and st_atime_usec hold the seconds and microseconds, respectively, of the system clock at the last time the object was read with io_read.

st_mtime and st_mtime_usec hold the second and microseconds, respectively, of the system clock at the last time the object was written with io_write.

Other appropriate operations may update the atime and the mtime as well; both the file and socket interfaces specify such operations.

st_ctime and st_ctime_usec hold the seconds and microseconds, respectively, of the system clock at the last time permanent meta-data associated with the object was changed.  The exact operations which couse such an update are server-dependent, but must include the creation of the object.

The server is permitted to delay the actual update of these times until stat is called; before the server stores the times on permanent media (if it ever does so) it should update them if necessary.

st_blksize gives the optimal I/O size in bytes for io_read and io_write; users should endeavor to read and write amounts which are multiples of the optimal size, and to use offsets which are multiples of the optimal size

In addition, objects which are seekable should set st_size to the ``maximum correct value'' described above in the description of the O_APPEND flag.

The st_uid and st_gid fields are unrelated to the ``owner'' as described above for icky async I/O.

Users may find out the version of the server they are talking to by calling io_server_version; this should return strings and integers describing the version number of the server, as well as its name.

Mapped data
===========

Servers may optionally implement the io_map call.  The ports returned by io_map must implement the XP kernel interface and be suitable as arguments to vm_map.

Seekable objects must allow access from 0 to the ``maximum correct value'' described for O_APPEND.  Whether they provide access beyond such a point is server dependent; in addition, the meaning of such an object for a non-seekable object is server dependent.

File interface
**************

This chapter documents the interface for operating on files.

File overview
=============

The file interface is a superset of the I/O interface (see I/O interface).  Servers which provide the file interface are required to support the I/O interface as well.  All objects reachable in the filesystem are expected to provide the file interface, even if they do not contain data.  (The trivfs library make it easy to do so for ordinary sorts of cases.  See Trivfs library.)

The interface definitions for the file interface are found in <hurd/fs.defs>.

Files have various pieces of status information which are returned by io_stat (see Information queries).  Most of this status information can be directly changed by various calls in the file interface; some of it should vary implicitly as the contents of the file change.

Many of these calls have general rules associated with them describing how security and privilege should operate.  The diskfs (see Diskfs library) implements these rules for disk-based filesystems.  (Trivfs based servers generally have no need to implement these rules at all.) We hope to move the implementation of these rules to the fshelp library.

In special cases, there may be a reason to implement a different security check from that specified here, or to implement a call to do something slightly different.  But such cases must be carefully considered; make sure that you will not confuse blameless user programs through excessive cleverness.

If some operation cannot be implemented (for example, chauthor over ftp), then the call should return EOPNOTSUPP.  If it is merely difficult to implement a call, it is much better to figure out a way to implement it as a series of operations rather than returning errors to the user.

Changing status
===============

There are several RPC's avalaible for users to change much of the status information associated with a file.  (The information is returned by the io_stat rpc; Information queries.)

All these operations are restricted to root and the owner of the file. When attempted by another user, they should return EPERM.

The file_chown RPC changes the owner and group of the file.  Only root should be able to change the owner, and changing the group to a group the caller is not in should also be prohibited.  Violating either of these conditions should return EPERM.

The file_chauthor RPC changes the author of the file.  It should be legitimate to change the author to any value without restriction.

The file_chflags RPC changes the flags of the file.  It should be legitimate to change the flags to any value without restriction.  No standard meanings have been assigned to the flags yet, but we intend to do so.  Do not assume that the flags format we choose will map identically to that for some existing filesystem format.

The file_utimes RPC changes the atime and mtime of the file.  Making this call must cause the ctime to be updated as well, even if no actual change to either the mtime or the atime occurs.

The file_set_size RPC is special; not only does it change the status word specifing the size of the file, but it also changes the actual contents of the file.   If the file size is being reduced it should release secondary storage associated with the previous contents of the file.  If the file is being extended, the new region added to the file must be zero filled.  Unlike the other RPC's in this section, file_set_size should be permitted to any user who is allowed to write the file.

Program execution
=================

Execution of programs on the Hurd is done through file servers with the file_exec RPC.  The fileserver is expected to verify that the user is allowed to execute the file, make whatever modifications to the ports are necessary for setuid execution, and then invoke the standard execserver found on /servers/exec.

This section specifically addresses what file servers are expected to do, with minimal attention to the other parts of the process.

The file must be opened for execution; if it is not, EBADF should be returned In addition, at least one of the execute bits must be on.  A failure of this check should result in EACCES--not ENOEXEC.  It is not proper for the file server to ever respond ENOEXEC in response to the file_exec RPC.

If either the setuid or setgid bits are set, the server needs to construct a new authentication handle with the additional new id's. Then all the ports passed to file_exec need to be reauthenticated with the new handle.  If the fileserver is unable to make the new authentication handle (for example, because it is not running as root) it is not acceptable to return an error; in such a case the server should simply silently fail to implement the setuid/setgid semantics.

If the setuid/setgid transformation adds a new uid or gid to the user's authentication handle that was not previously present (as opposed to merely reordering them) then the EXEC_SECURE and EXEC_NEWTASK flags should both be added in the call to exec_exec.

The server then needs to open a new port onto the executed file which will not share any filepointers with the port the user passed in, opened with O_READ.  Finally, all the information (mutated appropriately for setuid/setgid) should be sent to the execserver with exec_exec. Whatever error code exec_exec returns should returned to the caller of file_exec.

File locking
============

FIXME: Implementing the flock call

File frobbing
=============

FIXME: Other active calls on files

Opening files
=============

FIXME: Looking up files in directories

Modifying directories
=====================

FIXME: Creating and deleting nodes

Notifications
=============

FIXME: File and directory change callbacks

Translators
===========

FIXME: How to set and get translators

Filesystem interface
********************

FIXME: interfaces supported to control file-servers

Socket interface
****************

FIXME: Interfaces used for manipulating sockets

Ports library
*************

FIXME: A library to manage port rights for servers

Iohelp library
**************

FIXME: A library to implement some common parts of the I/O and shared I/O interfaces

Fshelp library
**************

FIXME: A library to implement some common parts of the file interface

Pager library
*************

FIXME: A library to implement complex multi-threaded pagers

Diskfs library
**************

FIXME: A library to do almost all the work of implementing a disk-based filesystem

Trivfs library
**************

FIXME: A library to do the work of handling the file protocol for directory-less filesystems