path: root/hurd/documentation/translators.html

[[meta copyright="Copyright © 1998, 1999, 2007, 2008 Free Software Foundation,
Inc."]]

[[meta license="Verbatim copying and distribution of this entire article is
permitted in any medium, provided this notice is preserved."]]

[[meta title="Translators"]]

By Marcus Brinkmann.

<ul>
<li><a href="#concept" name="TOC_concept">Concept</a></li>
<li><a href="#examples" name="TOC_examples">Examples</a></li>
<li><a href="#actpas" name="TOC_actpas">Passive Translators, Active Translators</a></li>
<li><a href="#manage" name="TOC_manage">Managing Translators</a></li>
</ul>
<h3><a href="#TOC_concept" name="concept">Concept</a></h3>
<p>
Before we take a closer look at translators, let us consider regular
filesystems. A filesystem is store for a hierarchical tree of directories
and files. You access directories and files by a special character string,
the path. Furthermore, there are symbolic links to refer to one file at
several places in the tree, there are hard links to give one and the same
file several names. There are also special device files for communication
with the hardware device drivers of the kernel, and there are mount points
to include other stores in the directory tree. Then there are obscure
objects like fifos and hard links.</p>
<p>
Although these objects are very different, they share some common
properties, for example, they have all an owner and a group associated with
them as well as access rights (permissions). This information is written in
inodes. This is a actually a further commonality: Every object has exactly
one inode associated with it (hard links are somewhat special as they share
one and the same inode). Sometimes, the inode has further information
stored in it. For example, the inode can contain the target of a symbolic
link.</p>
<p>
However, these commonalities are usually not exploited in the
implementations, despite the common programming interface to them. All
inodes can be accessed through the standard POSIX calls, for example
<code>read()</code> and <code>write()</code>. For example, to add a new
object type (for example a new link type) to a common monolithic unix
kernel, you would need to modify the code for each filesystem
seperately.</p>
<p>
In the Hurd, things work differently. Although in the Hurd a special
filesystem server can exploit special properties of standard object types
like links (in the ext2 filesystem with fast links, for example), it has a
general interface to add such features without modifying existing code.</p>
<p>
The trick is to allow a program to be inserted between the actual content of
a file and the user accessing this file. Such a program is called a
translator, because it is able to process the incoming requests in many
different ways. In other words, a translator is a Hurd server which provides
the basic filesystem interface.</p>
<p>
Translators have very interesting properties. From the kernels point of
view, they are just another user process. This means, translators can be run
by any user. You don't need root priviligies to install or modify a
translator, you only need the access rights for the underlying inode the
translator is attached to. Many translators don't require an actual file to
operate, they can provide information by their own means. This is why
the information about translators is stored in the inode.</p>
<p>
Translators are responsible to serve all file system operations that involve
the inode they are attached to. Because they are not restricted to the usual
set of objects (device file, link etc), they are free to return anything
that makes sense to the programmer. One could imagine a translator that
behaves like a directory when accessed by <code>cd</code> or
<code>ls</code> and at the same time behaves like a file when accessed by
<code>cat</code>.</p>
<h3><a href="#TOC_examples" name="examples">Examples</a></h3>
<h4>Mount Points</h4>
<p>
A mount point can be seen as an inode that has a special translator attached
to it. Its purpose would be to translate filesystem operations on the mount
point in filesystem operations on another store, let's say, another
partition.</p>
<p>
Indeed, this is how filesystems are implemented under the Hurd. A
filesystem is a translator. This translator takes a store as its argument,
and is able to serve all filesystem operations transparently.</p>
<h4>Device Files</h4>
<p>
There are many different device files, and in systems with a monolithical
kernel, they are all provided by the kernel itself. In the Hurd, all device
files are provided by translators. One translator can provide support for
many similar device files, for example all hard disk partitions. This way,
the number of actual translators needed is quite small. However, note that
for each device file accessed, a seperate translator task is started.
Because the Hurd is heavily multi threaded, this is very cheap.</p>
<p>
When hardware is involved, a translator usually starts to communicate with
the kernel to get the data from the hardware. However, if no hardware access
is necessary, the kernel does not need to be involved. For example,
<code>/dev/zero</code> does not require hardware access, and can therefore
be implemented completely in user space.</p>
<h4>Symbolic Links</h4>
<p>
A symbolic link can be seen as a translator. Accesing the symbolic link
would start up the translator, which would forward the request to the
filesystem that contains the file the link points to.</p>
<p>
However, for better performance, filesystems that have native support
for symbolic links can take advantage of this feature and implement
symbolic links differently. Internally, accessing a symbolic link would not
start a new translator process. However, to the user, it would still look
as if a passive translator is involved (see below for an explanation what a
passsive translator is).</p>
<p>
Because the Hurd ships with a symlink translator, any filesystem server that
provides support for translators automatically has support for symlinks (and
firmlinks, and device files etc)! This means, you can get a working
filesystem very fast, and add native support for symlinks and other features
later.</p>
<h3><a href="#TOC_actpas" name="actpas">Passive Translators, Active Translators</a></h3>
<p>
There are two types of translators, passive and active. They are really
completely different things, so don't mix them up, but they have a close
relation to each other.</p>
<h4>Active Translators</h4>
<p>
An active translator is a running translator process, as introduced above.
You can set and remove active translators using the
<code>settrans -a</code></a>
command. The <code>-a</code> option is necessary to tell
<code>settrans</code> that you want to modify the active translator.</p>
<p>
The <code>settrans</code> command takes three kind of arguments. First, you
can set options for the <code>settrans</code> command itself, like
<code>-a</code> to modify the active translator. Then you set the inode you
want to modify. Remember that a translator is always associated with an
inode in the directory hierarchy. You can only modify one inode at a time.
If you do not specify any more arguments, <code>settrans</code> will try to
remove an existing translator. How hard it tries depends on the force
options you specify (if the translator is in use by any process, you will
get "device or resource busy" error message unless you force it to go away).</p>
<p>
But if you specify further arguments, it will be interpreted as a command
line to run the translator. This means, the next argument is the filename of
the translator executable. Further arguments are options to the translator,
and not to the <code>settrans</code> command.</p>
<p>
For example, to mount an ext2fs partition, you can run
<code>settrans -a -c /mnt /hurd/ext2fs /dev/hd2s5</code>. The
<code>-c</code> option will create the mount point for you if it doesn't
exist already. This does not need to be a directory, by the way. To unmount,
you would try <code>settrans -a /mnt</code>.</p>
<h4>Passive Translators</h4>
<p>
A passive translator is set and modified with the same syntax as the active
translator (just leave away the <code>-a</code>, so everything said above is
true for passive translators, too. However, there is a difference: passive
translators are not yet started.</p>
<p>
This makes sense, because this is what you usually want. You don't want the
partition mounted unless you really access files on this partition. You
don't want to bring up the network unless there is some traffic and so
on.</p>
<p>
Instead, the first time the passive translator is accessed, it is
automatically read out of the inode and an active translator is started on
top of it using the command line that was stored in the inode. This is
similar to the Linux automounter functionality. However, it does not come as
an additional bonus that you have to set up manually, but an integral part of
the system. So, setting passive translators defers starting the translator
task until you really need it. By the way, if the active translator dies for
some reason, the next time the inode is accessed the translator is
restarted.</p>
<p>
There is a further difference: active translators can die or get lost. As
soon as the active translator process is killed (for example, because you
reboot the machine) it is lost forever. Passive translators are not transient
and stay in the inode during reboots until you modify them with the
<code>settrans</code> program or delete the inodes they are attached to.
This means, you don't need to maintain a configuration file with your mount
points.</p>
<p>
One last point: Even if you have set a passive translator, you can still
set a different active translator. Only if the translator is automatically
started because there was no active translator the time the inode was
accessed the passive translator is considered.</p>
<h3><a href="#TOC_manage" name="manage">Managing Translators</a></h3>
<p>
As mentioned above, you can use
<code>settrans</code></a>
to set and alter passive and active translators. There are a lot of options
to change the behaviour of <code>settrans</code> in case something goes
wrong, and to conditionalize its action. Here are some common usages:</p>
<ul><li><code>settrans -c /mnt /hurd/ext2fs /dev/hd2s5</code> mounts a
partition, the translator will stay across reboots.</li>
<li><code>settrans -a /mnt /hurd/ext2fs ~/dummy.fs</code> mounts a
filesystem inside a data file, the translator will go away if it dies.</li>
<li><code>settrans -fg /nfs-data</code> forces a translator to go away.</li>
</ul>
<p>
You can use the <code>showtrans</code></a>
command to see if a translator is attached to an inode. This will only show
you the passive translator though.</p>
<p>
You can change the options of an active (filesystem) translator with
<code>fsysopts</code> without actually restarting it. This is very
convenient. For example, you can do what is called "remounting a
partition read-only" under Linux simply by running <code>fsysopts
/mntpoint --readonly</code>. The running active translator
will change its behaviour according to your request if possible.
<code>fsysopts /mntpoint</code> without a parameter shows you the current
settings.</p>
<h4>Examples</h4>
<p>
I recommend that you start by reading the <code>/bin/mount</code> command,
it is only a small script. Because setting filesystem translators is
similar to mounting partitions, you can easily grasp the concept this way.
Make a file system image with <code>dd if=/dev/zero of=dummy.fs bs=1024k
count=8; mke2fs dummy.fs</code> and "mount" it with <code>settrans -c dummy
/hurd/ext2fs `pwd`/dummy.fs</code>. Note that the translator is not started
yet, no new <code>ext2fs</code> process is running (verify with <code>ps
Aux</code>). Check that everything is correct using <code>showtrans</code></p>
<p>
Now type <code>ls dummy</code> and you will notice the short delay that
occurs while the translator is started. After that, there will be no more
delays accessing dummy. Under Linux, one would say that you automounted a
loop file system. Check with <code>ps Aux</code> that there is an <code>ext2fs
dummy</code> process up and running now. Now put some files into the new
directory. Try to make the filesystem read-only with <code>fsysopts</code>.
Note how further write attempts fail now. Try to kill the active translator
with <code>settrans -g</code>.</p>
<p>
You should have some understanding of what is going on now. Now remember
that this was only <em>one</em> special server, the Hurd ext2fs server.
There are many more server in the <code>hurd</code> directory. Some of them
are for filesystems. Some are needed for file system features like links.
Some are needed for device files. Some are useful for networking. Imagine
"mounting" an FTP Server with <code>settrans</code> and downloading files
simply with the standard <code>cp</code> command. Or editing your web sites
with <code>emacs /ftp/homepage.my.server.org/index.html</code>!</p>