From a3bbfa6bc8a3abd8dd3c56fd8fe975028f6fa005 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Thu, 6 Nov 2008 12:56:40 +0100 Subject: Integrate whatis/translator.html. --- hurd/documentation.mdwn | 2 + hurd/documentation/translators.html | 236 +++++++++++++++++++++++++++++ hurd/translator.mdwn | 2 + whatis/translator.html | 290 ------------------------------------ 4 files changed, 240 insertions(+), 290 deletions(-) create mode 100644 hurd/documentation/translators.html delete mode 100644 whatis/translator.html diff --git a/hurd/documentation.mdwn b/hurd/documentation.mdwn index a32bc84f..be0874e7 100644 --- a/hurd/documentation.mdwn +++ b/hurd/documentation.mdwn @@ -24,6 +24,8 @@ is included in the section entitled * [[*The_Hurd*|hurd-talk]], a presentation by Marcus Brinkmann. + * A document about *[[translators]]* by Marcus Brinkmann. + * [[*A_Critique_of_the_GNU_Hurd_Multi-server_Operating_System*|critique]], an analysis of the GNU Hurd on GNU Mach system, written by Neal Walfield and Marcus Brinkmann. diff --git a/hurd/documentation/translators.html b/hurd/documentation/translators.html new file mode 100644 index 00000000..4e47a9c0 --- /dev/null +++ b/hurd/documentation/translators.html @@ -0,0 +1,236 @@ +[[meta copyright="Copyright © 1998, 1999, 2007, 2008 Free Software Foundation, +Inc."]] + +[[meta license="Verbatim copying and distribution of this entire article is +permitted in any medium, provided this notice is preserved."]] + +[[meta title="Translators"]] + +By Marcus Brinkmann. + + +

Concept

+

+Before we take a closer look at translators, let us consider regular +filesystems. A filesystem is store for a hierarchical tree of directories +and files. You access directories and files by a special character string, +the path. Furthermore, there are symbolic links to refer to one file at +several places in the tree, there are hard links to give one and the same +file several names. There are also special device files for communication +with the hardware device drivers of the kernel, and there are mount points +to include other stores in the directory tree. Then there are obscure +objects like fifos and hard links.

+

+Although these objects are very different, they share some common +properties, for example, they have all an owner and a group associated with +them as well as access rights (permissions). This information is written in +inodes. This is a actually a further commonality: Every object has exactly +one inode associated with it (hard links are somewhat special as they share +one and the same inode). Sometimes, the inode has further information +stored in it. For example, the inode can contain the target of a symbolic +link.

+

+However, these commonalities are usually not exploited in the +implementations, despite the common programming interface to them. All +inodes can be accessed through the standard POSIX calls, for example +read() and write(). For example, to add a new +object type (for example a new link type) to a common monolithic unix +kernel, you would need to modify the code for each filesystem +seperately.

+

+In the Hurd, things work differently. Although in the Hurd a special +filesystem server can exploit special properties of standard object types +like links (in the ext2 filesystem with fast links, for example), it has a +general interface to add such features without modifying existing code.

+

+The trick is to allow a program to be inserted between the actual content of +a file and the user accessing this file. Such a program is called a +translator, because it is able to process the incoming requests in many +different ways. In other words, a translator is a Hurd server which provides +the basic filesystem interface.

+

+Translators have very interesting properties. From the kernels point of +view, they are just another user process. This means, translators can be run +by any user. You don't need root priviligies to install or modify a +translator, you only need the access rights for the underlying inode the +translator is attached to. Many translators don't require an actual file to +operate, they can provide information by their own means. This is why +the information about translators is stored in the inode.

+

+Translators are responsible to serve all file system operations that involve +the inode they are attached to. Because they are not restricted to the usual +set of objects (device file, link etc), they are free to return anything +that makes sense to the programmer. One could imagine a translator that +behaves like a directory when accessed by cd or +ls and at the same time behaves like a file when accessed by +cat.

+

Examples

+

Mount Points

+

+A mount point can be seen as an inode that has a special translator attached +to it. Its purpose would be to translate filesystem operations on the mount +point in filesystem operations on another store, let's say, another +partition.

+

+Indeed, this is how filesystems are implemented under the Hurd. A +filesystem is a translator. This translator takes a store as its argument, +and is able to serve all filesystem operations transparently.

+

Device Files

+

+There are many different device files, and in systems with a monolithical +kernel, they are all provided by the kernel itself. In the Hurd, all device +files are provided by translators. One translator can provide support for +many similar device files, for example all hard disk partitions. This way, +the number of actual translators needed is quite small. However, note that +for each device file accessed, a seperate translator task is started. +Because the Hurd is heavily multi threaded, this is very cheap.

+

+When hardware is involved, a translator usually starts to communicate with +the kernel to get the data from the hardware. However, if no hardware access +is necessary, the kernel does not need to be involved. For example, +/dev/zero does not require hardware access, and can therefore +be implemented completely in user space.

+

Symbolic Links

+

+A symbolic link can be seen as a translator. Accesing the symbolic link +would start up the translator, which would forward the request to the +filesystem that contains the file the link points to.

+

+However, for better performance, filesystems that have native support +for symbolic links can take advantage of this feature and implement +symbolic links differently. Internally, accessing a symbolic link would not +start a new translator process. However, to the user, it would still look +as if a passive translator is involved (see below for an explanation what a +passsive translator is).

+

+Because the Hurd ships with a symlink translator, any filesystem server that +provides support for translators automatically has support for symlinks (and +firmlinks, and device files etc)! This means, you can get a working +filesystem very fast, and add native support for symlinks and other features +later.

+

Passive Translators, Active Translators

+

+There are two types of translators, passive and active. They are really +completely different things, so don't mix them up, but they have a close +relation to each other.

+

Active Translators

+

+An active translator is a running translator process, as introduced above. +You can set and remove active translators using the +settrans -a +command. The -a option is necessary to tell +settrans that you want to modify the active translator.

+

+The settrans command takes three kind of arguments. First, you +can set options for the settrans command itself, like +-a to modify the active translator. Then you set the inode you +want to modify. Remember that a translator is always associated with an +inode in the directory hierarchy. You can only modify one inode at a time. +If you do not specify any more arguments, settrans will try to +remove an existing translator. How hard it tries depends on the force +options you specify (if the translator is in use by any process, you will +get "device or resource busy" error message unless you force it to go away).

+

+But if you specify further arguments, it will be interpreted as a command +line to run the translator. This means, the next argument is the filename of +the translator executable. Further arguments are options to the translator, +and not to the settrans command.

+

+For example, to mount an ext2fs partition, you can run +settrans -a -c /mnt /hurd/ext2fs /dev/hd2s5. The +-c option will create the mount point for you if it doesn't +exist already. This does not need to be a directory, by the way. To unmount, +you would try settrans -a /mnt.

+

Passive Translators

+

+A passive translator is set and modified with the same syntax as the active +translator (just leave away the -a, so everything said above is +true for passive translators, too. However, there is a difference: passive +translators are not yet started.

+

+This makes sense, because this is what you usually want. You don't want the +partition mounted unless you really access files on this partition. You +don't want to bring up the network unless there is some traffic and so +on.

+

+Instead, the first time the passive translator is accessed, it is +automatically read out of the inode and an active translator is started on +top of it using the command line that was stored in the inode. This is +similar to the Linux automounter functionality. However, it does not come as +an additional bonus that you have to set up manually, but an integral part of +the system. So, setting passive translators defers starting the translator +task until you really need it. By the way, if the active translator dies for +some reason, the next time the inode is accessed the translator is +restarted.

+

+There is a further difference: active translators can die or get lost. As +soon as the active translator process is killed (for example, because you +reboot the machine) it is lost forever. Passive translators are not transient +and stay in the inode during reboots until you modify them with the +settrans program or delete the inodes they are attached to. +This means, you don't need to maintain a configuration file with your mount +points.

+

+One last point: Even if you have set a passive translator, you can still +set a different active translator. Only if the translator is automatically +started because there was no active translator the time the inode was +accessed the passive translator is considered.

+

Managing Translators

+

+As mentioned above, you can use +settrans +to set and alter passive and active translators. There are a lot of options +to change the behaviour of settrans in case something goes +wrong, and to conditionalize its action. Here are some common usages:

+ +

+You can use the showtrans +command to see if a translator is attached to an inode. This will only show +you the passive translator though.

+

+You can change the options of an active (filesystem) translator with +fsysopts without actually restarting it. This is very +convenient. For example, you can do what is called "remounting a +partition read-only" under Linux simply by running fsysopts +/mntpoint --readonly. The running active translator +will change its behaviour according to your request if possible. +fsysopts /mntpoint without a parameter shows you the current +settings.

+

Examples

+

+I recommend that you start by reading the /bin/mount command, +it is only a small script. Because setting filesystem translators is +similar to mounting partitions, you can easily grasp the concept this way. +Make a file system image with dd if=/dev/zero of=dummy.fs bs=1024k +count=8; mke2fs dummy.fs and "mount" it with settrans -c dummy +/hurd/ext2fs `pwd`/dummy.fs. Note that the translator is not started +yet, no new ext2fs process is running (verify with ps +Aux). Check that everything is correct using showtrans

+

+Now type ls dummy and you will notice the short delay that +occurs while the translator is started. After that, there will be no more +delays accessing dummy. Under Linux, one would say that you automounted a +loop file system. Check with ps Aux that there is an ext2fs +dummy process up and running now. Now put some files into the new +directory. Try to make the filesystem read-only with fsysopts. +Note how further write attempts fail now. Try to kill the active translator +with settrans -g.

+

+You should have some understanding of what is going on now. Now remember +that this was only one special server, the Hurd ext2fs server. +There are many more server in the hurd directory. Some of them +are for filesystems. Some are needed for file system features like links. +Some are needed for device files. Some are useful for networking. Imagine +"mounting" an FTP Server with settrans and downloading files +simply with the standard cp command. Or editing your web sites +with emacs /ftp/homepage.my.server.org/index.html!

diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index 889f02a6..1a987b09 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -40,6 +40,8 @@ Also there is an [[writing/example]] about how to write a simple translator. See some [[examples]] about how to use translators. +Marcus Brinkmann has written a document about [[documentation/translators]]. + # Existing Translators diff --git a/whatis/translator.html b/whatis/translator.html deleted file mode 100644 index 516d5049..00000000 --- a/whatis/translator.html +++ /dev/null @@ -1,290 +0,0 @@ - - - - - GNU Hurd - Free Software Foundation (FSF) - - - - - - - - - - -
-The GNU Hurd
-

-Whatis?
-Howto?
-

- -

- -Acknowledgements
- -

-

GNU Hurd

-

Translators

-

By Marcus Brinkmann

- -

Concept

-

-Before we take a closer look at translators, let us consider regular -filesystems. A filesystem is store for a hierarchical tree of directories -and files. You access directories and files by a special character string, -the path. Furthermore, there are symbolic links to refer to one file at -several places in the tree, there are hard links to give one and the same -file several names. There are also special device files for communication -with the hardware device drivers of the kernel, and there are mount points -to include other stores in the directory tree. Then there are obscure -objects like fifos and hard links.

-

-Although these objects are very different, they share some common -properties, for example, they have all an owner and a group associated with -them as well as access rights (permissions). This information is written in -inodes. This is a actually a further commonality: Every object has exactly -one inode associated with it (hard links are somewhat special as they share -one and the same inode). Sometimes, the inode has further information -stored in it. For example, the inode can contain the target of a symbolic -link.

-

-However, these commonalities are usually not exploited in the -implementations, despite the common programming interface to them. All -inodes can be accessed through the standard POSIX calls, for example -read() and write(). For example, to add a new -object type (for example a new link type) to a common monolithic unix -kernel, you would need to modify the code for each filesystem -seperately.

-

-In the Hurd, things work differently. Although in the Hurd a special -filesystem server can exploit special properties of standard object types -like links (in the ext2 filesystem with fast links, for example), it has a -general interface to add such features without modifying existing code.

-

-The trick is to allow a program to be inserted between the actual content of -a file and the user accessing this file. Such a program is called a -translator, because it is able to process the incoming requests in many -different ways. In other words, a translator is a Hurd server which provides -the basic filesystem interface.

-

-Translators have very interesting properties. From the kernels point of -view, they are just another user process. This means, translators can be run -by any user. You don't need root priviligies to install or modify a -translator, you only need the access rights for the underlying inode the -translator is attached to. Many translators don't require an actual file to -operate, they can provide information by their own means. This is why -the information about translators is stored in the inode.

-

-Translators are responsible to serve all file system operations that involve -the inode they are attached to. Because they are not restricted to the usual -set of objects (device file, link etc), they are free to return anything -that makes sense to the programmer. One could imagine a translator that -behaves like a directory when accessed by cd or -ls and at the same time behaves like a file when accessed by -cat.

-

Examples

-

Mount Points

-

-A mount point can be seen as an inode that has a special translator attached -to it. Its purpose would be to translate filesystem operations on the mount -point in filesystem operations on another store, let's say, another -partition.

-

-Indeed, this is how filesystems are implemented under the Hurd. A -filesystem is a translator. This translator takes a store as its argument, -and is able to serve all filesystem operations transparently.

-

Device Files

-

-There are many different device files, and in systems with a monolithical -kernel, they are all provided by the kernel itself. In the Hurd, all device -files are provided by translators. One translator can provide support for -many similar device files, for example all hard disk partitions. This way, -the number of actual translators needed is quite small. However, note that -for each device file accessed, a seperate translator task is started. -Because the Hurd is heavily multi threaded, this is very cheap.

-

-When hardware is involved, a translator usually starts to communicate with -the kernel to get the data from the hardware. However, if no hardware access -is necessary, the kernel does not need to be involved. For example, -/dev/zero does not require hardware access, and can therefore -be implemented completely in user space.

-

Symbolic Links

-

-A symbolic link can be seen as a translator. Accesing the symbolic link -would start up the translator, which would forward the request to the -filesystem that contains the file the link points to.

-

-However, for better performance, filesystems that have native support -for symbolic links can take advantage of this feature and implement -symbolic links differently. Internally, accessing a symbolic link would not -start a new translator process. However, to the user, it would still look -as if a passive translator is involved (see below for an explanation what a -passsive translator is).

-

-Because the Hurd ships with a symlink translator, any filesystem server that -provides support for translators automatically has support for symlinks (and -firmlinks, and device files etc)! This means, you can get a working -filesystem very fast, and add native support for symlinks and other features -later.

-

Passive Translators, Active Translators

-

-There are two types of translators, passive and active. They are really -completely different things, so don't mix them up, but they have a close -relation to each other.

-

Active Translators

-

-An active translator is a running translator process, as introduced above. -You can set and remove active translators using the -settrans -a -command. The -a option is necessary to tell -settrans that you want to modify the active translator.

-

-The settrans command takes three kind of arguments. First, you -can set options for the settrans command itself, like --a to modify the active translator. Then you set the inode you -want to modify. Remember that a translator is always associated with an -inode in the directory hierarchy. You can only modify one inode at a time. -If you do not specify any more arguments, settrans will try to -remove an existing translator. How hard it tries depends on the force -options you specify (if the translator is in use by any process, you will -get "device or resource busy" error message unless you force it to go away).

-

-But if you specify further arguments, it will be interpreted as a command -line to run the translator. This means, the next argument is the filename of -the translator executable. Further arguments are options to the translator, -and not to the settrans command.

-

-For example, to mount an ext2fs partition, you can run -settrans -a -c /mnt /hurd/ext2fs /dev/hd2s5. The --c option will create the mount point for you if it doesn't -exist already. This does not need to be a directory, by the way. To unmount, -you would try settrans -a /mnt.

-

Passive Translators

-

-A passive translator is set and modified with the same syntax as the active -translator (just leave away the -a, so everything said above is -true for passive translators, too. However, there is a difference: passive -translators are not yet started.

-

-This makes sense, because this is what you usually want. You don't want the -partition mounted unless you really access files on this partition. You -don't want to bring up the network unless there is some traffic and so -on.

-

-Instead, the first time the passive translator is accessed, it is -automatically read out of the inode and an active translator is started on -top of it using the command line that was stored in the inode. This is -similar to the Linux automounter functionality. However, it does not come as -an additional bonus that you have to set up manually, but an integral part of -the system. So, setting passive translators defers starting the translator -task until you really need it. By the way, if the active translator dies for -some reason, the next time the inode is accessed the translator is -restarted.

-

-There is a further difference: active translators can die or get lost. As -soon as the active translator process is killed (for example, because you -reboot the machine) it is lost forever. Passive translators are not transient -and stay in the inode during reboots until you modify them with the -settrans program or delete the inodes they are attached to. -This means, you don't need to maintain a configuration file with your mount -points.

-

-One last point: Even if you have set a passive translator, you can still -set a different active translator. Only if the translator is automatically -started because there was no active translator the time the inode was -accessed the passive translator is considered.

-

Managing Translators

-

-As mentioned above, you can use -settrans -to set and alter passive and active translators. There are a lot of options -to change the behaviour of settrans in case something goes -wrong, and to conditionalize its action. Here are some common usages:

-
  • settrans -c /mnt /hurd/ext2fs /dev/hd2s5 mounts a -partition, the translator will stay across reboots.
  • -
  • settrans -a /mnt /hurd/ext2fs ~/dummy.fs mounts a -filesystem inside a data file, the translator will go away if it dies.
  • -
  • settrans -fg /nfs-data forces a translator to go away.
  • -
-

-You can use the showtrans -command to see if a translator is attached to an inode. This will only show -you the passive translator though.

-

-You can change the options of an active (filesystem) translator with -fsysopts without actually restarting it. This is very -convenient. For example, you can do what is called "remounting a -partition read-only" under Linux simply by running fsysopts -/mntpoint --readonly. The running active translator -will change its behaviour according to your request if possible. -fsysopts /mntpoint without a parameter shows you the current -settings.

-

Examples

-

-I recommend that you start by reading the /bin/mount command, -it is only a small script. Because setting filesystem translators is -similar to mounting partitions, you can easily grasp the concept this way. -Make a file system image with dd if=/dev/zero of=dummy.fs bs=1024k -count=8; mke2fs dummy.fs and "mount" it with settrans -c dummy -/hurd/ext2fs `pwd`/dummy.fs. Note that the translator is not started -yet, no new ext2fs process is running (verify with ps -Aux). Check that everything is correct using showtrans

-

-Now type ls dummy and you will notice the short delay that -occurs while the translator is started. After that, there will be no more -delays accessing dummy. Under Linux, one would say that you automounted a -loop file system. Check with ps Aux that there is an ext2fs -dummy process up and running now. Now put some files into the new -directory. Try to make the filesystem read-only with fsysopts. -Note how further write attempts fail now. Try to kill the active translator -with settrans -g.

-

-You should have some understanding of what is going on now. Now remember -that this was only one special server, the Hurd ext2fs server. -There are many more server in the hurd directory. Some of them -are for filesystems. Some are needed for file system features like links. -Some are needed for device files. Some are useful for networking. Imagine -"mounting" an FTP Server with settrans and downloading files -simply with the standard cp command. Or editing your web sites -with emacs /ftp/homepage.my.server.org/index.html!

- -
- -Return to GNU's home page. -

- -Please send FSF & GNU inquiries & questions to - -gnu@gnu.org. -There are also other ways to -contact the FSF. -

- -Please send comments on these web pages to - -web-hurd@gnu.org, -send other questions to -gnu@gnu.org. -

-Copyright (C) 1998, 1999, 2007 Free Software Foundation, Inc., -59 Temple Place - Suite 330, Boston, MA 02111, USA -

-Verbatim copying and distribution of this entire article is -permitted in any medium, provided this notice is preserved.

-Updated: - -$Date$ $Author$ - -


-
- - - -- cgit v1.2.3