GNU Hurd

Translators

By Marcus Brinkmann

Concept
Examples
Passive Translators, Active Translators
Managing Translators

Concept

Before we take a closer look at translators, let us consider regular filesystems. A filesystem is store for a hierarchical tree of directories and files. You access directories and files by a special character string, the path. Furthermore, there are symbolic links to refer to one file at several places in the tree, there are hard links to give one and the same file several names. There are also special device files for communication with the hardware device drivers of the kernel, and there are mount points to include other stores in the directory tree. Then there are obscure objects like fifos and hard links.

Although these objects are very different, they share some common properties, for example, they have all an owner and a group associated with them as well as access rights (permissions). This information is written in inodes. This is a actually a further commonality: Every object has exactly one inode associated with it (hard links are somewhat special as they share one and the same inode). Sometimes, the inode has further information stored in it. For example, the inode can contain the target of a symbolic link.

However, these commonalities are usually not exploited in the implementations, despite the common programming interface to them. All inodes can be accessed through the standard POSIX calls, for example read() and write(). For example, to add a new object type (for example a new link type) to a common monolithic unix kernel, you would need to modify the code for each filesystem seperately.

In the Hurd, things work differently. Although in the Hurd a special filesystem server can exploit special properties of standard object types like links (in the ext2 filesystem with fast links, for example), it has a general interface to add such features without modifying existing code.

The trick is to allow a program to be inserted between the actual content of a file and the user accessing this file. Such a program is called a translator, because it is able to process the incoming requests in many different ways. In other words, a translator is a Hurd server which provides the basic filesystem interface.

Translators have very interesting properties. From the kernels point of view, they are just another user process. This means, translators can be run by any user. You don't need root priviligies to install or modify a translator, you only need the access rights for the underlying inode the translator is attached to. Many translators don't require an actual file to operate, they can provide information by their own means. This is why the information about translators is stored in the inode.

Translators are responsible to serve all file system operations that involve the inode they are attached to. Because they are not restricted to the usual set of objects (device file, link etc), they are free to return anything that makes sense to the programmer. One could imagine a translator that behaves like a directory when accessed by cd or ls and at the same time behaves like a file when accessed by cat.

Examples

Mount Points

A mount point can be seen as an inode that has a special translator attached to it. Its purpose would be to translate filesystem operations on the mount point in filesystem operations on another store, let's say, another partition.

Indeed, this is how filesystems are implemented under the Hurd. A filesystem is a translator. This translator takes a store as its argument, and is able to serve all filesystem operations transparently.

Device Files

There are many different device files, and in systems with a monolithical kernel, they are all provided by the kernel itself. In the Hurd, all device files are provided by translators. One translator can provide support for many similar device files, for example all hard disk partitions. This way, the number of actual translators needed is quite small. However, note that for each device file accessed, a seperate translator task is started. Because the Hurd is heavily multi threaded, this is very cheap.

When hardware is involved, a translator usually starts to communicate with the kernel to get the data from the hardware. However, if no hardware access is necessary, the kernel does not need to be involved. For example, /dev/zero does not require hardware access, and can therefore be implemented completely in user space.

Symbolic Links

A symbolic link can be seen as a translator. Accesing the symbolic link would start up the translator, which would forward the request to the filesystem that contains the file the link points to.

However, for better performance, filesystems that have native support for symbolic links can take advantage of this feature and implement symbolic links differently. Internally, accessing a symbolic link would not start a new translator process. However, to the user, it would still look as if a passive translator is involved (see below for an explanation what a passsive translator is).

Because the Hurd ships with a symlink translator, any filesystem server that provides support for translators automatically has support for symlinks (and firmlinks, and device files etc)! This means, you can get a working filesystem very fast, and add native support for symlinks and other features later.

Passive Translators, Active Translators

There are two types of translators, passive and active. They are really completely different things, so don't mix them up, but they have a close relation to each other.

Active Translators

An active translator is a running translator process, as introduced above. You can set and remove active translators using the settrans -a command. The -a option is necessary to tell settrans that you want to modify the active translator.

The settrans command takes three kind of arguments. First, you can set options for the settrans command itself, like -a to modify the active translator. Then you set the inode you want to modify. Remember that a translator is always associated with an inode in the directory hierarchy. You can only modify one inode at a time. If you do not specify any more arguments, settrans will try to remove an existing translator. How hard it tries depends on the force options you specify (if the translator is in use by any process, you will get "device or resource busy" error message unless you force it to go away).

But if you specify further arguments, it will be interpreted as a command line to run the translator. This means, the next argument is the filename of the translator executable. Further arguments are options to the translator, and not to the settrans command.

For example, to mount an ext2fs partition, you can run settrans -a -c /mnt /hurd/ext2fs /dev/hd2s5. The -c option will create the mount point for you if it doesn't exist already. This does not need to be a directory, by the way. To unmount, you would try settrans -a /mnt.

Passive Translators

A passive translator is set and modified with the same syntax as the active translator (just leave away the -a, so everything said above is true for passive translators, too. However, there is a difference: passive translators are not yet started.

This makes sense, because this is what you usually want. You don't want the partition mounted unless you really access files on this partition. You don't want to bring up the network unless there is some traffic and so on.

Instead, the first time the passive translator is accessed, it is automatically read out of the inode and an active translator is started on top of it using the command line that was stored in the inode. This is similar to the Linux automounter functionality. However, it does not come as an additional bonus that you have to set up manually, but an integral part of the system. So, setting passive translators defers starting the translator task until you really need it. By the way, if the active translator dies for some reason, the next time the inode is accessed the translator is restarted.

There is a further difference: active translators can die or get lost. As soon as the active translator process is killed (for example, because you reboot the machine) it is lost forever. Passive translators are not transient and stay in the inode during reboots until you modify them with the settrans program or delete the inodes they are attached to. This means, you don't need to maintain a configuration file with your mount points.

One last point: Even if you have set a passive translator, you can still set a different active translator. Only if the translator is automatically started because there was no active translator the time the inode was accessed the passive translator is considered.

Managing Translators

As mentioned above, you can use settrans to set and alter passive and active translators. There are a lot of options to change the behaviour of settrans in case something goes wrong, and to conditionalize its action. Here are some common usages:

settrans -c /mnt /hurd/ext2fs /dev/hd2s5 mounts a partition, the translator will stay across reboots.
settrans -a /mnt /hurd/ext2fs ~/dummy.fs mounts a filesystem inside a data file, the translator will go away if it dies.
settrans -fg /nfs-data forces a translator to go away.

You can use the showtrans command to see if a translator is attached to an inode. This will only show you the passive translator though.

You can change the options of an active (filesystem) translator with fsysopts without actually restarting it. This is very convenient. For example, you can do what is called "remounting a partition read-only" under Linux simply by running fsysopts /mntpoint --readonly. The running active translator will change its behaviour according to your request if possible. fsysopts /mntpoint without a parameter shows you the current settings.

Examples

I recommend that you start by reading the /bin/mount command, it is only a small script. Because setting filesystem translators is similar to mounting partitions, you can easily grasp the concept this way. Make a file system image with dd if=/dev/zero of=dummy.fs bs=1024k count=8; mke2fs dummy.fs and "mount" it with settrans -c dummy /hurd/ext2fs `pwd`/dummy.fs. Note that the translator is not started yet, no new ext2fs process is running (verify with ps Aux). Check that everything is correct using showtrans

Now type ls dummy and you will notice the short delay that occurs while the translator is started. After that, there will be no more delays accessing dummy. Under Linux, one would say that you automounted a loop file system. Check with ps Aux that there is an ext2fs dummy process up and running now. Now put some files into the new directory. Try to make the filesystem read-only with fsysopts. Note how further write attempts fail now. Try to kill the active translator with settrans -g.

You should have some understanding of what is going on now. Now remember that this was only one special server, the Hurd ext2fs server. There are many more server in the hurd directory. Some of them are for filesystems. Some are needed for file system features like links. Some are needed for device files. Some are useful for networking. Imagine "mounting" an FTP Server with settrans and downloading files simply with the standard cp command. Or editing your web sites with emacs /ftp/homepage.my.server.org/index.html!

Return to GNU's home page.

Please send FSF & GNU inquiries & questions to gnu@gnu.org. There are also other ways to contact the FSF.

Please send comments on these web pages to web-hurd@gnu.org, send other questions to gnu@gnu.org.

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.

Updated: $Date$ $Author$