diff options
Diffstat (limited to 'hurd')
41 files changed, 1009 insertions, 122 deletions
diff --git a/hurd/ada4hurd.mdwn b/hurd/ada4hurd.mdwn index c783e53b..e5ef1359 100644 --- a/hurd/ada4hurd.mdwn +++ b/hurd/ada4hurd.mdwn @@ -51,7 +51,7 @@ Ada4Hurd provides tools and examples to ease Ada development in Hurd. It is at a * Install the build dependencies as root - $ apt-get install gnat libopentoken4-dev libxmlada5-dev libasis2014-dev + $ apt install gnat libopentoken4-dev libxmlada5-dev libasis2014-dev * Build @@ -65,4 +65,4 @@ Ada4Hurd provides tools and examples to ease Ada development in Hurd. It is at a * Run netfs tests * In netfs\_base directory - $ make trans\_dbg\_on
\ No newline at end of file + $ make trans\_dbg\_on diff --git a/hurd/bootstrap.mdwn b/hurd/bootstrap.mdwn new file mode 100644 index 00000000..fbce3bc1 --- /dev/null +++ b/hurd/bootstrap.mdwn @@ -0,0 +1,311 @@ +[[!meta copyright="Copyright © 2020 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="System bootstrap"]] + +[[!tag open_issue_documentation]] <!-- Someone still needs to make a pass over +this text. --> + +[[!toc]] + +# State at the beginning of the bootstrap + +After initializing itself, GNU Mach sets up tasks for the various bootstrap +translators (which were loader by the GRUB bootloader). It notably makes +variables replacement on their command lines and boot script function calls (see +the details in `gnumach/kern/boot_script.c`). For instance, if the GRUB +bootloader has the following configuration: + + multiboot /boot/gnumach-1.8-486-dbg.gz root=device:hd1 console=com0 + module /hurd/ext2fs.static ext2fs --readonly \ + --multiboot-command-line='${kernel-command-line}' \ + --host-priv-port='${host-port}' \ + --device-master-port='${device-port}' \ + --exec-server-task='${exec-task}' -T typed '${root}' \ + '$(task-create)' '$(task-resume)' + module /lib/ld.so.1 exec /hurd/exec '$(exec-task=task-create)' + + +GNU Mach will first make the `$(task-create)` function calls, and thus create a +task for the ext2fs module and a task for the exec module (and store a port on +that task in the `exec-task` variable). + +It will then replace the variables (`${foo}`), i.e. + +* `${kernel-command-line}` with its own command line (`root=device:hd1 console=com0`), +* `${host-port}` with a reference to the GNU Mach host port, +* `${device-port}` with a reference to the GNU Mach device port, +* `${exec-task}` with a reference to the exec task port. +* `${root}` with `device:hd1` + +This typically results in: + + task loaded: ext2fs --readonly --multiboot-command-line=root="device:hd1 console=com0" --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd1 + task loaded: exec /hurd/exec + +(You will have noticed that `/hurd/exec` is not run directly, but through +`ld.so.1`: Mach only knows to run statically-linked ELF binaries, so we could +either load `/hurd/exec.static`, or load the dynamic loader `ld.so.1` and tell +it to load `/hurd/exec`) + +GNU Mach will eventually make the `$(task-resume)` function calls, and thus +resume the ext2fs task only. + +This starts a dance between the bootstrap processes: `ext2fs`, `exec`, `startup`, +`proc`, and `auth`. Indeed, there are a few dependencies between them: `exec` needs +`ext2fs` working to be able to start `startup`, `proc` and `auth`, and `ext2fs` needs to +register itself to `startup`, `proc` and `auth` so as to appear as a normal process, +running under uid 0. + +The base principle is that `ext2fs` has a nul bootstrap port set to while other +translators will have a non-nul bootstrap port, with which they will discuss. We +thus have a hierarchy between the bootstrap processes. `ext2fs` is at the root, +`exec` and `startup` are its direct children, and `auth` and `port` are direct +children of `startup`. + +Usually the bootstrap port is used when starting a translator, see +`fshelp_start_translator_long`: the parent translator starts the child and sets +its bootstrap port. The parent then waits for the child to call `fsys_startup` +on the bootstrap port, for the child to provide its control port, and for the +parent to provide the FS node that the child is translator for. + +For the bootstrap translators, the story is extended: + +* Between `ext2fs` and `startup`: `startup` additionally calls `fsys_init`, to +provide `ext2fs` with `proc` and `auth` ports. +* Between `startup` and `proc`: `proc` just calls `startup_procinit` to hand +over a `proc` port and get `auth` and `priv` ports. +* Between `startup` and `auth`: `auth` calls `startup_authinit` to hand over an +`auth` port and get a `proc` port, then calls `startup_essential_task` to notify +`startup` that the boot can proceed. +* For translators before `ext2fs`, the child calls `fsys_startup` to pass over +the control port of `ext2fs` (instead of its own control port, which is useless +for its parent). This is typically done in the `S_fsys_startup` stub, simply +forwarding it. The child also calls `fsys_init` to pass over the `proc` and +`auth` ports. Again, this is typically done in the `S_fsys_init` stub, simply +forwarding them. + +With that in mind, the dance between the bootstrap translators is happening as +described in the next sections. + +# Initialization of translators before ext2fs + +We plan to start pci-arbiter and rumpdisk translators before ext2fs. + +pci-arbiter's `main` function parses the arguments, and since it is given a disk +task port, it knows it is a bootstrap translator and thus initializes the +machdev library. `machdev_trivfs_init` resumes the disk task. + +rumpdisk's `main` function parses the arguments, and since it is given a FS task +port, it knows it is a bootstrap translator, and thus `machdev_trivfs_init` +resumes the FS task. + +# ext2fs initialization + +ext2fs's `main` function starts by calling `diskfs_init_main`. + +`diskfs_init_main` parses the ext2fs command line with `argp_parse`, to record +the parameters set up by the kernel. It makes sure to have a working stdout by +opening the Mach console. + +Since the multiboot command line is available, `diskfs_init_main` sets the +ext2fs bootstrap port to `MACH_PORT_NULL`: it is the bootstrap filesystem which +will be in charge of dancing with the exec and startup translator. + +`diskfs_init_main` then initializes the libdiskfs library and spawns a thread to +manage libdiskfs RPCs. + +ext2fs continues its initialization: creating a pager, opening the +hypermetadata, opening the root inode to be set as root by libdiskfs. + +ext2fs then calls `diskfs_startup_diskfs` to really run the startup, implemented +by the libdiskfs library. + +# libdiskfs bootstrap + +Since the bootstrap port is `MACH_PORT_NULL`, `diskfs_startup_diskfs` calls +`diskfs_start_bootstrap`. + +`diskfs_start_bootstrap` starts by creating a open port on itself for the +current and root directory, all other processes will inherit it. + +`diskfs_start_bootstrap` does have a port on the exec task, so it can dance with +it. It calls `start_execserver` that sets the bootstrap port of the exec task +to a port of the `diskfs_execboot_class`, and resumes the exec task. +`diskfs_start_bootstrap` then waits for `execstarted`. + +# exec bootstrap + +exec's `main` function starts and calls `task_get_bootstrap_port` to get +its bootstrap port and `getproc` to get a port on the proc translator (thus +`MACH_PORT_NULL` at this point since the proc translator is not started yet). + +exec initializes the trivfs library, and eventually calls `trivfs_startup` on +its bootstrap port. + +`trivfs_startup` creates a control port for the exec translator, and calls +`fsys_startup` on the bootstrap port to notify ext2fs that it is ready, give it +its exec control port, and get back a port on the underlying node for the exec +translator (we want to make it show up on `/servers/exec`). + +# libdiskfs taking back control + +`diskfs_execboot_fsys_startup` is thus called. It calls `dir_lookup` on +`/servers/exec` to return the underlying node for the exec translator, and +stores the `exec` control port in `diskfs_exec_ctl`. It can then signal `execstarted`. + +`diskfs_start_bootstrap` thus takes back control, It calls `fsys_getroot` on the +control port of exec, and uses `dir_lookup` and `file_set_translator` to attach +it to `/servers/exec`. + +`diskfs_start_bootstrap` then looks for which startup process to run. It may +be specified on the multiboot command line, but otherwise it will default to +`/hurd/startup`. + +Now that exec is up and running, the `startup` process can be created with +`exec_exec`. `diskfs_start_bootstrap` takes a lot of care in this: this is +the first unix-looking process, it notably inherits the root directory and +current working directory initialized above, it gets stdin/out/err on the mach +console. It is passed as bootstrap port a port from the `diskfs_control_class`. + +`diskfs_start_bootstrap` is complete, we are back to `diskfs_startup_diskfs`, +which checks whether ext2fs was given a bootstrap port, i.e. whether +the rumpdisk translator was started before ext2fs. If so, it +calls `diskfs_call_fsys_startup` which creates a new control port and passes +it do a call to `fsys_startup` on the bootstrap port, so rumpdisk gets access +to the ext2fs filesystem. Rumpdisk however does not return any `realnode` port, +since we are not exposing the ext2fs filesystem in rumpdisk, but rather the +converse. TODO: Rumpdisk forwards this `fsys_startup` call to pci-arbiter, so +the latter also gets access to the ext2fs filesystem. + +# startup + +startup's `main` function starts and calls `task_get_bootstrap_port` to get its +bootstrap port, i.e. the control port of ext2fs, and `fsys_getpriv` on it to get +the host privileged port and device master port. It +clears the bootstrap port so children do not inherit it. It sets itself up with +output on the Mach console, and wires itself against swapping. It requests +notification for ext2fs translator dying to detect it and print a warning in +case that happens during boot. It creates a `startup` port which it will get +RPCs on. + +startup can then complete the unixish initialization, and run `/hurd/proc` and +`/hurd/auth`, giving them as bootstrap port the `startup` port. + +# proc + +proc's `main` function starts. It initializes itself, and calls +`task_get_bootstrap_port` to get a port on startup. It can then call +`startup_procinit` on it to pass it the proc port that will represent the startup +task, and get ports on the auth server, the host privileged port, and device +master port. + +Eventually, proc calls `startup_essential_task` to tell startup that it is +ready. + +# auth + +auth's `main` function starts. It creates the initial root auth handle (all +permissions allowed). It calls `task_get_bootstrap_port` to get a port on +startup. It can then call `startup_authinit` on it to pass the initial root auth +handle, and get a port on the proc server. It can then register itself to proc. + +Eventually, auth calls `startup_essential_task` to tell startup that it is ready. + +# startup getting back control + +startup notices initialization of auth and proc from `S_startup_procinit` and +`S_startup_authinit`. Once both have been called, it calls `launch_core_servers`. + +`launch_core_servers` starts by calling `startup_procinit_reply` to actually +reply to the `startup_procinit` RPC with a port on auth. + +`launch_core_servers` then tells proc that the bootstrap processes are +important, and how they relate to each other. + +`launch_core_servers` then calls `install_as_translator` to show up in the +filesystem on `/servers/startup`. + +`launch_core_servers` calls `startup_authinit_reply` to actually reply to the +`startup_authinit` RPC with a port on proc. + +`launch_core_servers` eventually calls `fsys_init` on its bootstrap port, to +give ext2fs the proc and auth ports. + +diskfs' `diskfs_S_fsys_init` thus gets called. It first replies to startup, so +startup is not stuck in its `fsys_init` call and not able to reply to RPCs. From +then on, startup will be watching for `startup_essential_task` calls from the +various bootstrap processes. + +# libdiskfs taking back control + +In diskfs' `diskfs_S_fsys_init`, diskfs now knows that proc and auth are ready, +and can call `exec_init` on the exec port. + +# exec getting initialized + +exec's `S_exec_init` gets called from the `exec_init` call from ext2fs. Exec can +register itself with proc, and eventually call `startup_essential_task` to tell +startup that it is ready. + +# back to libdiskfs initialization + +It also calls `fsys_init` +on its bootstrap port, i.e. rumpdisk. + +# rumpdisk getting initialized + +rumpdisk's `trivfs_S_fsys_init` gets called from the `fsys_init` call from +ext2fs. It calls `fsys_init` on its bootstrap port. + +# pci-arbiter getting initialized + +pci-arbiter's `trivfs_S_fsys_init` gets called from the `fsys_init` call from +rumpdisk. + +It gets the root node of ext2fs, sets all common ports, and install +pci-arbiter in the ext2fs FS as translator for `/servers/bus/pci`. + +It eventually calls `startup_essential_task` to tell startup that it is ready, +and requests shutdown notifications. + +# back to rumpdisk initialization + +It gets the root node of ext2fs, sets all common ports, and install +rumpdisk in the ext2fs FS as translator for `/dev/disk`. + +It eventually calls `startup_essential_task` to tell startup that it is ready, +and requests shutdown notifications. + +# back to libdiskfs initialization + +It initializes the default proc and auth ports to be given to processes. + +It calls `startup_essential_task` on the startup port to tell startup that +it is ready. + +Eventually, it calls `_diskfs_init_completed` to finish its initialization, and +notably call `startup_request_notification` to get notified by startup when the +system is shutting down. + +# startup monitoring bootstrap progress + +As mentioned above, the different essential tasks (pci-arbiter, rumpdisk, ext2fs, proc, auth, exec) +call `startup_essential_task` when they are ready. startup's +`S_startup_essential_task` function thus gets called each time, and startup +records each of them as essential, monitoring their death to crash the whole +system. + +Once all of proc, auth, exec have called `startup_essential_task`, startup +replies to their respective RPCs, so they actually start working altogether. It +also calls `init_stdarrays` which sets the initial values of the standard exec data, and `frob_kernel_process` to plug the kernel task into the picture. +It eventually calls `launch_something`, which "launches +something", which by default is `/libexec/runsystem`, but if that can not be +found, launches a shell instead, so the user can fix it. diff --git a/hurd/building.mdwn b/hurd/building.mdwn index 31d909e5..63c33498 100644 --- a/hurd/building.mdwn +++ b/hurd/building.mdwn @@ -24,8 +24,8 @@ Building the Hurd requires the *build-essential* and *fakeroot* packages, their dependencies and additional packages that are specified by the source hurd package: - # apt-get install build-essential fakeroot - # apt-get build-dep hurd + # apt install build-essential fakeroot + # apt build-dep hurd ## ... on non-Debian systems @@ -41,9 +41,9 @@ git](http://savannah.gnu.org/git/?group=hurd): ... or (if you are working on a Debian system) the ones that are used for the [current Debian hurd package](http://packages.debian.net/source/unstable/hurd): - $ apt-get source hurd + $ apt source hurd -Please see the Debian [[FAQ]] before using `apt-get source`. +Please see the Debian [[FAQ]] before using `apt source`. The unpacked source tree is around 20 MiB, and the build tree (configured with `--disable-profile`) is around 100 MiB. @@ -93,6 +93,12 @@ or `/local/`, so your current Hurd servers will be replaced. To install to a different location, specify `--prefix=PREFIX` as `configure` parameter, e.g. `--prefix=/usr` (as done when having a real `/usr`). +To build acpi: + + $ make acpi + +You may need to install necessary acpi headers (`libacpica-dev` package in Debian based distro). + By default profiling versions of all the libraries and code are generated but this is useless in most of the cases, so we disable them by specifying `--disable-profile` on `configure`'s command line. diff --git a/hurd/coding_style.mdwn b/hurd/coding_style.mdwn index cc1e3cf3..93a40ba2 100644 --- a/hurd/coding_style.mdwn +++ b/hurd/coding_style.mdwn @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation]] +The Hurd repository is following the [GNU Coding Standards](https://www.gnu.org/prep/standards/). + Some coding style comments that are specific to Hurd systems. [[!toc]] diff --git a/hurd/dde/guide.mdwn b/hurd/dde/guide.mdwn index dd36f1f5..b6cf7753 100644 --- a/hurd/dde/guide.mdwn +++ b/hurd/dde/guide.mdwn @@ -58,11 +58,11 @@ Download the packages for offline installation: $ cd /mnt - $ apt-get -c etc/apt/apt.conf.offline update + $ apt -c etc/apt/apt.conf.offline update - $ apt-get -c etc/apt/apt.conf.offline build-dep hurd gnumach + $ apt -c etc/apt/apt.conf.offline build-dep hurd gnumach - $ apt-get -c etc/apt/apt.conf.offline install git-core build-essential libpciaccess-dev libpcap0.8-dev hurd-dev zlib1g-dev + $ apt -c etc/apt/apt.conf.offline install git-core build-essential libpciaccess-dev libpcap0.8-dev hurd-dev zlib1g-dev Get DDE code: @@ -117,9 +117,9 @@ so we can boot into Hurd to do the actual work. Once there, install the packages previously downloaded (again as root): - $ apt-get build-dep hurd gnumach + $ apt build-dep hurd gnumach - $ apt-get install git-core build-essential libpciaccess-dev libpcap0.8-dev hurd-dev zlib1g-dev + $ apt install git-core build-essential libpciaccess-dev libpcap0.8-dev hurd-dev zlib1g-dev Make sure we can build stuff as normal user: diff --git a/hurd/debugging/glibc.mdwn b/hurd/debugging/glibc.mdwn index 10a41503..1b7e6ab1 100644 --- a/hurd/debugging/glibc.mdwn +++ b/hurd/debugging/glibc.mdwn @@ -44,24 +44,26 @@ testsuite, use: To save even more build, stop the build after configure has run, and then you can restart the build of only libc.so and libc.a with: - cd build-tree/hurd-i386-libc - make lib + make -C build-tree/hurd-i386-libc lib or of only libc.so with: - make objdir=$PWD/build-tree/hurd-i386-libc $PWD/build-tree/hurd-i386-libc/libc.so + make -C build-tree/hurd-i386-libc objdir=$PWD/build-tree/hurd-i386-libc $PWD/build-tree/hurd-i386-libc/libc.so or of the whole tree with: - cd build-tree/hurd-i386-libc - make + make -C build-tree/hurd-i386-libc or of just one subdir with for instance: - make subdir=libpthread -C libpthread ..=../ objdir=$PWD/build-tree/hurd-i386-libc + make -C htl subdir=htl ..=../ objdir=$PWD/build-tree/hurd-i386-libc (note that most subdirs need libc.so built) +Similarly, you can run the testsuite of a single directory the same way: + + make check -C htl subdir=htl ..=../ objdir=$PWD/build-tree/hurd-i386-libc + --- In some cases, printing to stdout/stderr is problematic. One can use a kernel @@ -75,6 +77,16 @@ If your `mach_traps.h` does not have the declaration, use: extern void mach_print(const char *s); +If you can't use the libc-provided `mach_print` for some reason, you can use as last resort: + + asm("\ + my_mach_print:\n\ + mov $-30,%eax\n\ + lcall $7,$0\n\ + ret\n\ + "); + extern void my_mach_print(const char *s); + This call does not support any formatting. You can use this kind of helper to format a message before passing to gnumach: diff --git a/hurd/discussion.mdwn b/hurd/discussion.mdwn new file mode 100644 index 00000000..56802b75 --- /dev/null +++ b/hurd/discussion.mdwn @@ -0,0 +1,13 @@ +[[!meta copyright="Copyright © 2018 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +MutoShack: + +Hey, does "media references" mean screenshots, or is it synonymous with "media appearances"? I don't know if a new page is needed, or if it's pointing to the wrong (non-existing) page. diff --git a/hurd/documentation.mdwn b/hurd/documentation.mdwn index 330ea13b..f095cf26 100644 --- a/hurd/documentation.mdwn +++ b/hurd/documentation.mdwn @@ -1,5 +1,5 @@ [[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, -2009, 2011, 2012, 2015 Free Software Foundation, Inc."]] +2009, 2011, 2012, 2015, 2019, Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,7 +12,7 @@ is included in the section entitled [[!meta title="Hurd Documentation"]] # Introductory Material - * [[What_Is_the_GNU_Hurd]] + * [[What_Is_the_GNU_Hurd?|What_Is_the_GNU_Hurd]] * [[Advantages of the Hurd|Advantages]] * [[The Hurd FAQ|FAQ]] * [[*The_Hurd_and_Linux*|/hurd-and-linux]], a comment by Richard Stallman. @@ -76,7 +76,7 @@ is included in the section entitled [[Position_paper_*Improving_Usability_via_Access_Decomposition_and_Policy*|ng/position_paper]] Neal Walfield and Marcus Brinkmann give an overview about how a future, subsequent system may be architected. - * [*Generalizing mobility for the Hurd*](http://users.student.lth.se/cs07fh9/2009-hammar-hurd-mobility.pdf), + * [*Generalizing mobility for the Hurd*](https://pdfs.semanticscholar.org/4cce/9abb177cc58c199e13b82d498f37010c2bfc.pdf), a thesis written by Carl Fredrik Hammar, investigates the mobility aspect of stores and how it can be generalized and used for other applications. diff --git a/hurd/documentation/translator_primer.mdwn b/hurd/documentation/translator_primer.mdwn index 92a1d5f9..073d5e07 100644 --- a/hurd/documentation/translator_primer.mdwn +++ b/hurd/documentation/translator_primer.mdwn @@ -84,7 +84,7 @@ What you do here is setting up the translator /hurd/hostmux on ftp: and passing Now that we can access ftp.gnu.org transparently, let's mount a remote ISO file: - $ settrans -c mnt /hurd/iso9660fs ftp://ftp.gnu.org/old-gnu/gnu-f2/hurd-F2-main.iso + $ settrans -c mnt /hurd/iso9660fs $PWD/ftp://ftp.gnu.org/old-gnu/gnu-f2/hurd-F2-main.iso $ ls mnt/ It is interesting to note that since the ISO9660 format is indexed, ftpfs does not have to download the whole ISO file, it merely fetches what iso9660fs requests. diff --git a/hurd/glibc.mdwn b/hurd/glibc.mdwn index 4b5e8d38..8e330aef 100644 --- a/hurd/glibc.mdwn +++ b/hurd/glibc.mdwn @@ -27,18 +27,18 @@ glibc. This should be working as per the following: $ mkdir -p /tmp/build/src $ cp -a /usr/src/glibc /tmp/build/src/ $ unset CFLAGS - $ /tmp/build/src/glibc/scripts/build-many-glibcs.py /tmp/build checkout - $ /tmp/build/src/glibc/scripts/build-many-glibcs.py /tmp/build host-libraries - $ /tmp/build/src/glibc/scripts/build-many-glibcs.py /tmp/build compilers i686-gnu + $ /tmp/build/src/glibc/scripts/build-many-glibcs.py --shallow /tmp/build checkout + $ /tmp/build/src/glibc/scripts/build-many-glibcs.py --strip /tmp/build host-libraries + $ /tmp/build/src/glibc/scripts/build-many-glibcs.py --strip /tmp/build compilers i686-gnu $ /tmp/build/src/glibc/scripts/build-many-glibcs.py /tmp/build glibcs i686-gnu Currently the master branch builds that way without any testsuite issue. -# Building +To save some disk space, after the compilers stage you can remove src/mpc, src/mpfr, src/binutils, src/linux. -One of the tests really put boxes on its knees: +Build logs are available in `/tmp/build/logs` - $ echo "tests-unsupported += test-lfs" >> sysdeps/mach/hurd/i386/Makefile +# Building One can build libc this way: @@ -52,3 +52,7 @@ One can build libc this way: One can run tests with the new libc by hand: $ ./testrun.sh ~/test + +One can build by hand some target with e.g.: + + $ make $PWD/htl/libpthread.so -C ../htl subdir=htl objdir=$PWD ..=../ diff --git a/hurd/interface/fs/19.mdwn b/hurd/interface/fs/19.mdwn index 86625d44..2a50d3e0 100644 --- a/hurd/interface/fs/19.mdwn +++ b/hurd/interface/fs/19.mdwn @@ -23,7 +23,10 @@ License|/fdl]]."]]"""]] Read entries from the directory. Each entry is identified by an index number starting at 0 and running through the file. This call fetches `nentries` (or any convenient number if `nentries` is -1) entries starting at `entry`, -returning an array of struct directs in `data`. The number of entries +returning a series of struct dirent in `data`. +Note that due to the variable-size `d_name` field, `d_reclen` has to be used to +jump from one struct dirent to the other. +The number of entries successfully read is returned in `amount`. If `entry` is bigger than the index of the last entry, then 0 is returned in `amount`. If `bufsize` is nonzero, never return more than `bufsize` bytes of data regardless. diff --git a/hurd/libports.mdwn b/hurd/libports.mdwn index b0a0f6d5..c2c39226 100644 --- a/hurd/libports.mdwn +++ b/hurd/libports.mdwn @@ -16,8 +16,8 @@ ports|microkernel/mach/port]]. It is documented in the [[Reference_Manual]]. Mach ports to the functionality the Hurd needs, that is, it is not meant to provide an interface independently of the underlying [[microkernel]]. -*libports* does not itself depend on *[[libthreads]]*, but the appropriate -threading hooks are used if present, that is if *[[libthreads]]* is used by +*libports* does not itself depend on *[[/libpthread]]*, but the appropriate +threading hooks are used if present, that is if *[[/libpthread]]* is used by another component. diff --git a/hurd/libthreads.mdwn b/hurd/libthreads.mdwn deleted file mode 100644 index aa429d81..00000000 --- a/hurd/libthreads.mdwn +++ /dev/null @@ -1,39 +0,0 @@ -[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -`libthreads` a.k.a. C threads. - -**Note**: since Hurd migrated to [[libpthread]] as threading library, -the development and usage of libthreads has been discontinued. - - - -# Internals - -## Threading Model - -libthreads has a 1:1 threading model. - - -## Threads' Death - -A thread's death doesn't actually free the thread's stack (and maybe not the -associated Mach ports either). That's because there's no way to free the stack -after the thread dies (because the thread of control is gone); the stack needs -to be freed by something else, and there's nothing convenient to do it. There -are many ways to make it work. - -However, it isn't really a leak, because the unfreed resources do get used for -the next thread. So the issue is that the shrinkage of resource consumption -never happens, but it doesn't grow without bounds; it just stays at the maximum -even if the current number of threads is lower. - -The same issue exists in [[libpthread]]. diff --git a/hurd/porting/guidelines.mdwn b/hurd/porting/guidelines.mdwn index fd533fd7..624f7fd5 100644 --- a/hurd/porting/guidelines.mdwn +++ b/hurd/porting/guidelines.mdwn @@ -132,6 +132,8 @@ If you get Bad File Descriptor error when trying to read from a file (or accessi <http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html> +Also see <https://eklitzke.org/path-max-is-tricky> and <https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html> + Every unconditionalized use of `PATH_MAX`, `MAX_PATH` or `MAXPATHLEN` is a POSIX incompatibility. If there is no upper limit on the length of a path (as its the case for GNU), this symbol is not defined in any header file. Instead, you need to either use a different implementation that does not rely on the length of a string or use `sysconf()` to query the length at runtime. If `sysconf()` returns -1, you have to use `realloc()` to allocate the needed memory dynamically. Usually it is thus simpler to just use dynamic allocation. Sometimes the amount is actually known. Else, a geometrically growing loop can be used: for instance, see [Pulseaudio patch](http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=5;filename=patch-pulse;att=1;bug=522100). Note that in some cases there are GNU extensions that just work fine: when the `__GLIBC__` macro is defined, `getcwd()` calls can be just replaced by `get_current_dir_name()` calls. Note: constants such as `_POSIX_PATH_MAX` are only the minimum required value @@ -140,10 +142,10 @@ for a potential corresponding `PATH_MAX` macro. They are not a replacement for Note 2: Yes, some POSIX functions such as `realpath()` actually assume that `PATH_MAX` is defined. This is a bug of the POSIX standard, which got fixed in -POSIX 2001, in which one can simply pass `NULL` to get a dynamically +POSIX 2008, in which one can simply pass `NULL` to get a dynamically allocated buffer. One can thus use: - #if _POSIX_VERSION >= 200112 || defined(__GLIBC__) + #if _POSIX_VERSION >= 200809 || defined(__GLIBC__) char *path = realpath(orig, NULL); #else char path[PATH_MATH]; @@ -158,7 +160,7 @@ Same rationale as `PATH_MAX`. There is no limit on the number of arguments. Same rationale as `PATH_MAX`. There is no limit on the number of iovec items. -## <a name="MAXHOSTNAMELEN_tt_"> `MAXHOSTNAMELEN` </a> +## <a name="MAXHOSTNAMELEN_tt_"> `MAXHOSTNAMELEN`, `HOST_NAME_MAX` </a> Same as `PATH_MAX`. When you find a `gethostname()` function, which acts on a static buffer, you can replace it with Neal's [xgethostname function](http://walfield.org/pub/people/neal/xgethostname/) which returns the hostname as a dynamic buffer. For example: diff --git a/hurd/porting/system_api_limitations.mdwn b/hurd/porting/system_api_limitations.mdwn index 1615ccc0..5fe13fdb 100644 --- a/hurd/porting/system_api_limitations.mdwn +++ b/hurd/porting/system_api_limitations.mdwn @@ -22,8 +22,5 @@ These are the known system API limits that have porting implications. **_[\#47998](http://bugs.debian.org/47998): `msgget` IPC not implemented_** -**_[[nice() doesn't work|open_issues/nice_vs_mach_thread_priorities]]_**. - **_[\#187391](http://bugs.debian.org/187391): libc0.3-dev: `sockaddr_un.sun_path` can't be assigned a `const char *` when compiling with g++_**<br />**breaks:** fam, gail<br />**status:** maybe this should be in [[PortingIssues]] (see _long_ bug log) -**_[\#190367](http://bugs.debian.org/190367): libc0.3-dev: `fcntl` `F_GETLK` not implemented (`ENOSYS`)_**<br />**breaks:** gnome-session (and others) from running<br />**error:** misc lock-related errors diff --git a/hurd/rump.mdwn b/hurd/rump.mdwn new file mode 100644 index 00000000..ddde657f --- /dev/null +++ b/hurd/rump.mdwn @@ -0,0 +1,65 @@ +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag stable_URL]] + + * [[community/gsoc/project ideas/driver glue code]] + + * [[open issues/user-space device drivers]] + + * [[open issues/device drivers and io systems]] + +--- + +The rump kernels provide existing real world drivers from netbsd. +Since [[DDE]] no longer seems like a promising approach to get drivers +for the Hurd, it appears that rump kernels are the best alternative. +It already does the hard work of providing an environment where the +foreign drivers can run, and offers the additional benefit of being +externally maintained. Rump also offers the necessary facilities for +running all drivers in separate userspace processes, which is more +desirable than drivers running in the microkernel. + +A rump kernel is a minimal and portable NetBSD kernel running in +userspace. Rump kernels provide drivers for modern hard drives, sound +cards, usb support, and a TCP/IP stack. Instead of re-inventing and +maintaining drivers ourselves, we can re-use the existing NetBSD +drivers. + +Hurd developers have enabled experimental support for modern hard +drives with a rump kernel. We call it rumpdisk, and you can try it in +the [[Debian GNU/Hurd image|hurd/running/qemu]]. + +As of May 2023, Hurd users are having good success with it in qemu +environments and some are using it on real hardware! + +We do hope to use rump kernels for usb support, sound support (this +was working at some point), and possibly a new TCP/IP stack, but work +has not completed on those projects. + +# Documentation + + * <http://www.fixup.fi/misc/usenix-login-2015/login_oct15_02_kantee.pdf> + + This is an an opinion paper that explains why operating systems need compartmentalized kernel drivers. + + * <https://github.com/rumpkernel/wiki/wiki/Tutorial:-Getting-started> + + A tutorial introduction for those interested in using and deploying rump kernels. + + * <https://core.ac.uk/display/41816390> + + "User space approach to audio device driving on UNIX-like systems" by Robert Millan Hernandez. + + +# Source Code + + * <https://github.com/rumpkernel> diff --git a/hurd/running/Guix.mdwn b/hurd/running/Guix.mdwn new file mode 100644 index 00000000..64f9d0e7 --- /dev/null +++ b/hurd/running/Guix.mdwn @@ -0,0 +1,19 @@ +[[!meta title="Guix"]] + +GNU/Hurd support has been integrated in Guix. + +--- +# QEMU Image +[[!inline pages=hurd/running/Guix/qemu_image raw=yes feeds=no]] + +--- +# Documentation + +As Hurd support is integrated in Guix, the official documentation (<https://guix.gnu.org/en/manual/devel/>) also works for Hurd. + +Guix has even support in its configuration language for creating Hurd VMs from a running Guix system (<https://guix.gnu.org/en/manual/devel/en/guix.html#The-Hurd-in-a-Virtual-Machine>). + +--- +# Status + +At the time of writing, the official Guix manual says that "This configuration is experimental and under development. The easiest way for you to give it a try is by setting up an instance of hurd-vm-service-type on your GNU/Linux machine (see hurd-vm-service-type). See Contributing, on how to help!" (<https://guix.gnu.org/en/manual/devel/en/guix.html#GNU-Distribution>). diff --git a/hurd/running/Guix/qemu_image.mdwn b/hurd/running/Guix/qemu_image.mdwn new file mode 100644 index 00000000..52985c6f --- /dev/null +++ b/hurd/running/Guix/qemu_image.mdwn @@ -0,0 +1,14 @@ +[//]: # ([[meta copyright="Copyright © 2011, 2012, 2014, 2016 Free Software Foundation, Inc."]]) + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Guix's QEMU Image"]] + +There is a QEMU image with [[Guix GNU/Hurd|guix]] pre-installed available +at <https://ci.guix.gnu.org/search/latest/image?query=spec:images+status:success+system:x86_64-linux+hurd-barebones.qcow2>. diff --git a/hurd/running/chroot.mdwn b/hurd/running/chroot.mdwn index eac67282..0f5ec88f 100644 --- a/hurd/running/chroot.mdwn +++ b/hurd/running/chroot.mdwn @@ -24,7 +24,7 @@ It can be a good idea to put the chroot on a separate translator, for instance: Debootstrap should be able to build the content: - # debootstrap sid chroot + # debootstrap --keyring=/usr/share/keyrings/debian-ports-archive-keyring.gpg --extra-suites=unreleased sid chroot http://deb.debian.org/debian-ports/ # Tricks diff --git a/hurd/running/cloud.mdwn b/hurd/running/cloud.mdwn index 736a7113..3d0d37ef 100644 --- a/hurd/running/cloud.mdwn +++ b/hurd/running/cloud.mdwn @@ -15,4 +15,4 @@ It is possible to run the Hurd as a KVM-based OpenStack cloud instance. [[For the time being|open_issues/virtio]], you'll have to avoid using virtio drivers, and use emulated hardware instead: - $ glance image-create --property hw_disk_bus=ide --property hw_cdrom_bus=ide --property hw_vif_model=rtl8139 --disk-format raw --container-format bare --name gnu-hurd --copy-from https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd.img + $ glance image-create --property hw_disk_bus=ide --property hw_cdrom_bus=ide --property hw_vif_model=e1000 --disk-format raw --container-format bare --name gnu-hurd --copy-from https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd.img diff --git a/hurd/running/debian/CrossInstall.mdwn b/hurd/running/debian/CrossInstall.mdwn index c7a099c6..26cd77af 100644 --- a/hurd/running/debian/CrossInstall.mdwn +++ b/hurd/running/debian/CrossInstall.mdwn @@ -17,9 +17,9 @@ Next we create a useful mountpoint and mount the partition. ### <a name="Retrieving_CrossHurd"> Retrieving CrossHurd </a> -Unless you don't run Debian GNU/Linux download it from <http://packages.debian.org/crosshurd>, or simply apt-get the package from Testing or Unstable. Avoid using the version from Stable since it probably is outdated. In case of problems, make sure to try the Unstable version before reporting the issue. +Unless you don't run Debian GNU/Linux download it from <http://packages.debian.org/crosshurd>, or simply apt the package from Testing or Unstable. Avoid using the version from Stable since it probably is outdated. In case of problems, make sure to try the Unstable version before reporting the issue. - # apt-get install crosshurd + # apt install crosshurd ### <a name="Cross_installing"> Cross installing </a> diff --git a/hurd/running/debian/DebianAptOffline.mdwn b/hurd/running/debian/DebianAptOffline.mdwn index 9596040d..f97e5148 100644 --- a/hurd/running/debian/DebianAptOffline.mdwn +++ b/hurd/running/debian/DebianAptOffline.mdwn @@ -24,11 +24,11 @@ As root on the internet connected OS: # mount /dev/DEBIAN_GNU_HURD_PARTITON /mnt # cd /mnt - # apt-get -c etc/apt/apt.conf.offline {update, upgrade, install foo, etc.} + # apt -c etc/apt/apt.conf.offline {update, upgrade, install foo, etc.} Then, reboot into your Debian GNU/Hurd installation and as root, run: - # apt-get {update, upgrade, install foo, etc.} + # apt {update, upgrade, install foo, etc.} ## If you _cannot_ mount your Debian GNU/Hurd partition under another OS. @@ -47,7 +47,7 @@ From the remote sytem, as any user, run: $ cd myhurd $ tar -xf myhurdsconf.tar $ mkdir -p var/lib/apt/lists/partial var/cache/apt/archives/partial tmp - $ apt-get -c etc/apt/apt.conf.offline {update, upgrade, install foo, etc.} + $ apt -c etc/apt/apt.conf.offline {update, upgrade, install foo, etc.} $ tar cf myhurdsconf.tar etc/apt/{apt.conf.offline,sources.list} var/ Copy _myhurdsconf.tar_ back to your Debian GNU/Hurd system. @@ -59,4 +59,4 @@ Finally, from your Debian GNU/Hurd installation as the root user: # tar -xf myhurdsconf.tar # mv var/cache/apt/archives/*.deb /var/cache/apt/archives/ # mv var/lib/apt/lists/*_* /var/lib/apt/lists/ - # apt-get {update, upgrade, install foo, etc.} + # apt {update, upgrade, install foo, etc.} diff --git a/hurd/running/debian/MediaPressKitDiscuss.mdwn b/hurd/running/debian/MediaPressKitDiscuss.mdwn index 2bd97290..05e1761a 100644 --- a/hurd/running/debian/MediaPressKitDiscuss.mdwn +++ b/hurd/running/debian/MediaPressKitDiscuss.mdwn @@ -71,6 +71,6 @@ I think another active process for tracking recent news (if it doesn't already e Here are some interesting urls from [this issue](http://www.debian.org/News/weekly/2003/03/) of the Debian Weekly news: -**Debian Presentations.** Wolfgang Borgert was [looking](http://lists.debian.org/debian-devel-0301/msg00991.html) for a set of slides on dpkg, apt-get and debconf. Javier Fern�ndez-Sanguino Pe�a [intends](http://lists.debian.org/debian-devel-0301/msg01022.html) to provide a 'presentations' section in the [Debian Documentation Project](http://cvs.debian.org/ddp/?cvsroot=debian-doc) (DDP) and has already created an [archive](http://dat.etsit.upm.es/~jfs/debian/www/ddp/slides/) of slides. Whilst the Debian web site does link to [talks](http://www.debian.org/events/talks) given by developers and some [sample slides](http://www.debian.org/events/materials/slides/), it is difficult to gather this information and publish it in a homogeneous way. Talks should be reported to <events@debianNOSPAM.org> and forwarded to him. +**Debian Presentations.** Wolfgang Borgert was [looking](http://lists.debian.org/debian-devel-0301/msg00991.html) for a set of slides on dpkg, apt and debconf. Javier Fern�ndez-Sanguino Pe�a [intends](http://lists.debian.org/debian-devel-0301/msg01022.html) to provide a 'presentations' section in the [Debian Documentation Project](http://cvs.debian.org/ddp/?cvsroot=debian-doc) (DDP) and has already created an [archive](http://dat.etsit.upm.es/~jfs/debian/www/ddp/slides/) of slides. Whilst the Debian web site does link to [talks](http://www.debian.org/events/talks) given by developers and some [sample slides](http://www.debian.org/events/materials/slides/), it is difficult to gather this information and publish it in a homogeneous way. Talks should be reported to <events@debianNOSPAM.org> and forwarded to him. -- [[Main/GrantBow]] - 22 Jan 2003 diff --git a/hurd/running/debian/after_install.mdwn b/hurd/running/debian/after_install.mdwn index d3d32a6f..927d05f1 100644 --- a/hurd/running/debian/after_install.mdwn +++ b/hurd/running/debian/after_install.mdwn @@ -11,7 +11,7 @@ typing a boring arcane. There are Debian-specific scripts that may help you. See [[GRUB]]'s page for this. -# Setup `apt-get` +# Setup `apt Installing packages without having a network connection is described [[DebianAptOffline]]. diff --git a/hurd/running/debian/qemu_image.mdwn b/hurd/running/debian/qemu_image.mdwn index 6ae16d8b..9984ac33 100644 --- a/hurd/running/debian/qemu_image.mdwn +++ b/hurd/running/debian/qemu_image.mdwn @@ -22,16 +22,27 @@ Usage: $ wget https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd.img.tar.gz $ tar -xz < debian-hurd.img.tar.gz - $ kvm -m 1G -drive cache=writeback,file=$(echo debian-hurd-*.img) -no-reboot -net user,hostfwd=tcp:127.0.0.1:2222-:22 + $ kvm -m 1G -drive cache=writeback,file=$(echo debian-hurd-*.img) -no-reboot -net user,hostfwd=tcp:127.0.0.1:2222-:22 -net nic,model=e1000 -* Login as root (the root password is empty) +* Log in as root (the root password is empty) * Set up a root password with `passwd` +* update the system with `apt update && apt upgrade` + +* Log in as demo (the demo password is empty) +* Set up a demo password with `passwd` + +* You can also create another non-root user with `adduser <username>` +* and set the non-root user password with `passwd <username>` +* and add the non-root user to the sudo group via `gpasswd -a <user> sudo` + +* logout via `logout` + Optionally you may use `--curses` to keep your keyboard layout. If need be modprobe kvm_amd, kvm intel and kvm to get kvm support (which is much, much faster). Note that if you do not have a command named `kvm`, you can try something across the lines of: - $ qemu-system-i386 --enable-kvm -drive cche=writeback,file=$(echo debian-hurd-*.img) -net user,hostfwd=tcp:127.0.0.1:2222-:22 + $ qemu-system-i386 --enable-kvm -drive cche=writeback,file=$(echo debian-hurd-*.img) -net user,hostfwd=tcp:127.0.0.1:2222-:22 -net nic,model=e1000 Or, if your machine does not allow for KVM acceleration, omit `--enable-kvm` from the command. diff --git a/hurd/running/debian/status.mdwn b/hurd/running/debian/status.mdwn index 95e48edc..cf3592e7 100644 --- a/hurd/running/debian/status.mdwn +++ b/hurd/running/debian/status.mdwn @@ -1,4 +1,4 @@ Debian GNU/Hurd is currently an official, non-releasing Debian port. I.e., there is no testing or stable distribution. - - [Build daemon/archive status](http://unstable.buildd.net/buildd/hurd-i386_stats) - - [Number of registered users](http://buildd.net/cgi/archvote.phtml) + - [Build daemon/archive status](https://buildd.debian.org/status/architecture.php?a=hurd-i386&suite=sid) + - [Number of registered users](https://popcon.debian.org/stat/sub-hurd-i386.png) diff --git a/hurd/running/distrib.mdwn b/hurd/running/distrib.mdwn index 357d840a..5d12f8ec 100644 --- a/hurd/running/distrib.mdwn +++ b/hurd/running/distrib.mdwn @@ -14,6 +14,7 @@ There are several GNU distributions that are built on the Hurd. If you develop a ###Working distributions of GNU/Hurd: * [[Debian]] +* [[Guix]] ###GNU/Hurd distributions in early stages of development: diff --git a/hurd/running/live_cd.mdwn b/hurd/running/live_cd.mdwn index 4ff5a791..85cc8f9e 100644 --- a/hurd/running/live_cd.mdwn +++ b/hurd/running/live_cd.mdwn @@ -4,13 +4,23 @@ A less recent Live CD can be found at <http://teythoon.cryptobitch.de/hurd/livecd/hurd-live-install-1273300101.iso.xz>. -It can be run with qemu via +Arch Hurd also offers a QEMU image. It can be run with QEMU via + + $ wget https://files.archhurd.org/iso/2018.09.28/archhurd-2018.09.28.img.xz + $ xz -d archhurd-2018.09.28.img.xz + $ kvm -m 1G -drive cache=writeback,file=archhurd-2018.09.28.img + +The .img file is useful for running Arch Hurd in a virtual machine. For hardware installations, + use the Live CD. When using Arch Hurd in QEMU, the command 'hurd-halt' safely stops the +emulation (removing the need for '-no-reboot') + +To install the Live CD ISO file in QEMU (instead of using the .img file): $ wget http://teythoon.cryptobitch.de/hurd/livecd/hurd-live-install-1273300101.iso.xz $ xz -d hurd-live-install-1273300101.iso.xz $ qemu -m 512 -cdrom hurd-live-install-1273300101.iso -These [[!wikipedia LiveCD]]s should be useful for those who want to try out the +The [[!wikipedia LiveCD]] should be useful for those who want to try out the Hurd before they commit to installing it on their hard disks. In addition to that, the bootable Hurd CDs should enable us to have a native installer instead of relying on Linux. diff --git a/hurd/running/qemu.mdwn b/hurd/running/qemu.mdwn index b9afdde4..c56292c8 100644 --- a/hurd/running/qemu.mdwn +++ b/hurd/running/qemu.mdwn @@ -24,6 +24,56 @@ You can use the following images to give the Hurd a try. [[!inline pages=hurd/running/debian/qemu_image raw=yes feeds=no]] +#### Trying out rumpdisk + +[[Rump kernels|hurd/rump]] provide new modern drivers for the Hurd. +We refer to rumpdisk as a rump kernel that provides drivers for modern +hard drives, SSDs, etc. The Rump kernels' integration into the Hurd +are still somewhat experimental, but they seem to work fairly well on +bleeding edge Debian. + +Once you have your latest qemu Debian GNU/Hurd image running, then you +can try the rumpdisk (be sure to pass "-m 2GB" or more). First, +add these sources to your /etc/apt/sources.list + + deb http://deb.debian.org/debian-ports unstable main + deb-src http://deb.debian.org/debian unstable main + deb http://deb.debian.org/debian-ports unreleased main + +Then, upgrade to the bleeding edge Debian GNU/Hurd: + + # apt update + # apt upgrade --without-new-pkgs + # apt dist-upgrade + +Now test to see if the rump kernel works before you make the change +permanent. Manually tweak your /boot/grub/grub.cfg like so: + + # multiboot /boot/gnumach-1.8-486.gz root=part:2:device:hd0 console=com0 + multiboot /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 noide + +and your /etc/fstab + + #/dev/hd0s2 / ext2 defaults 0 1 + /dev/wd0s2 / ext2 defaults 0 1 + #/dev/hd0s1 none swap sw 0 0 + /dev/wd0s1 none swap sw 0 0 + #/dev/hd2 /media/cdrom0 iso9660 noauto 0 0 + /dev/wd2 /media/cdrom0 iso9660 noauto 0 0 + +Now you can poweroff your machine, reboot, and start using the +rumpdisk! You can make these changes permanent by tweaking +/etc/default/grub and telling it to use rumpdisk: + + GRUB_CMDLINE_GNUMACH="noide" + +Then update your grub: + + # update-grub + +Check that "noide" does appear in your /boot/grub/grub.cfg. + + ## Arch Hurd Live CD [[!inline pages=hurd/running/live_cd raw=yes feeds=no]] @@ -50,7 +100,7 @@ volunteers and may not have been tested extensively. ## Debian Installer -Instructions for creating a qemu image from the install CDs from debian installer can be found in the README alongside the d-i Hurd images: <http://people.debian.org/~sthibault/hurd-i386/installer/cdimage/> +Instructions for creating a qemu image from the install CDs from debian installer can be found in the README alongside the d-i Hurd images: <https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/> --- # KVM acceleration @@ -60,7 +110,7 @@ Check if your CPU supports kvm: $ egrep '^flags.*(vmx|svm)' /proc/cpuinfo #### If you don't have hardware support (slow): - $ apt-get install qemu + $ apt install qemu Do not enable kernel-kqemu, as that assumes some particular behavior from the guest kernel, which we are reluctant to artificially add to gnumach. @@ -68,7 +118,7 @@ If QEMU with KVM is not available, [[Virtualbox]] reportedly has better performance. #### If you have hardware support (recommended): - $ apt-get install qemu-kvm + $ apt install qemu-kvm $ modprobe kvm Intel VTx/VTd: Enable Intel kvm in the BIOS @@ -119,7 +169,7 @@ First off you will need to create a disk image using `qemu-img`. I have set mine Next you will want to start up QEMU and begin the installation process. - $ qemu -m 1G -drive cache=writeback,file=hd0.img -cdrom debian-7.0-hurd-i386-NETINST-1.iso -net nic,model=rtl8139 -net user + $ qemu -m 1G -drive cache=writeback,file=hd0.img -cdrom debian-7.0-hurd-i386-NETINST-1.iso -net nic,model=e1000 -net user Now at his point do the regular install using `hd0` as your harddrive. Partition it and install the base system. @@ -167,7 +217,7 @@ Once you have finished installing the base system (might take some time) the sys Starting qemu/qemu-kvm: - $ kvm -m 1G -net nic -net user,hostfwd=tcp::5555-:22 -drive cache=writeback,file=hd0.img -vga vmware + $ kvm -m 1G -net nic,model=e1000 -net user,hostfwd=tcp::5555-:22 -drive cache=writeback,file=hd0.img -vga vmware vmsvga_value_write: guest runs Linux. Note: See below on port forwarding in the networking section. @@ -252,13 +302,13 @@ If you are on [[Debian GNU/Hurd|debian]], you can even use [[debian/DHCP]]. To get ssh working: - # apt-get install random-egd openssh-server (Similarly for telnet if preferred) + # apt install openssh-server (Similarly for telnet if preferred) (See also <http://www.nongnu.org/qemu/qemu-doc.html#SEC32>.) Outgoing internet connections should just work then. Testing it can be difficult with a minimal installation, -but `apt-get update` should work after you have filled out +but `apt update` should work after you have filled out `/etc/apt/sources.list`. After that you should be able to install other network packages, but note that `ping` doesn't work with QEMU's user-networking stack. @@ -288,7 +338,7 @@ This is the recommended way to work with a Command Line Interface (CLI) since al a) with ssh (assuming you have installed openssh-server) - $ kvm -m 1G -net nic -net user,hostfwd=tcp::5555-:22 -drive cache=writeback,file=hd0.img & + $ kvm -m 1G -net nic,model=e1000 -net user,hostfwd=tcp::5555-:22 -drive cache=writeback,file=hd0.img & Logging in to the running Hurd: @@ -305,7 +355,7 @@ Copying files: b) with telnet (assuming you have installed a telnet server, like telnetd) - $ kvm -m 1G -net nic -net user,hostfwd=tcp::5556-:23 -drive cache=writeback,file=hurd-install.qemu & + $ kvm -m 1G -net nic,model=e1000 -net user,hostfwd=tcp::5556-:23 -drive cache=writeback,file=hurd-install.qemu & Logging in to the running Hurd: @@ -346,7 +396,7 @@ Now it is time to start-up your QEMU Hurd system and get networking going in the **Important:** Remember you may need to use the `-M isapc` or `-isa` flag if using an older version of the gnumach package. - $ qemu -m 1G -drive cache=writeback,file=hd0.img -cdrom debian-K9-hurd-i386-CD1.iso -fda floppy.img -boot a -net nic -net tap + $ qemu -m 1G -drive cache=writeback,file=hd0.img -cdrom debian-K9-hurd-i386-CD1.iso -fda floppy.img -boot a -net nic,model=e1000 -net tap Once you have logged in as `root` run the `pfinet` translator with values that apply to your network. Think of your QEMU client as another computer in your network. @@ -355,14 +405,16 @@ Once you have logged in as `root` run the `pfinet` translator with values that a That should do it! Do not forget to edit/update `/etc/resolv.conf` to get DNS working. --- -# Multiboot +# Booting Hurd without grub, using qemu's multiboot support See "Linux/Multiboot boot specific" section on QEMU manpage. Get the multiboot modules. Either extract them from the disk image, or, download: - $ wget http://people.debian.org/~sthibault/hurd-i386/installer/cdimage/current/{gnumach.gz,ext2fs.static,ld.so.1} + $ wget https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/gnumach.gz + $ wget https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/ext2fs.static + $ wget https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/exec.static Generally, these files need to correspond to the ones in the disk image, so don't forget to keep them up to date. @@ -372,18 +424,15 @@ you'll get told: *qemu: linux kernel too old to load a ram disk*. $ qemu [...] \ > --kernel gnumach \ + > --append 'root=device:hd0s1' \ > --initrd \ - > 'ext2fs.static --multiboot-command-line=${kernel-command-line} --host-priv-port=${host-port} --device-master-port=${device-port} --exec-server-task=${exec-task} -T typed device:hd0s1 $(task-create) $(task-resume)',\ - > 'ld.so.1 /hurd/exec $(exec-task=task-create)' + > 'ext2fs.static --multiboot-command-line=${kernel-command-line} --host-priv-port=${host-port} --device-master-port=${device-port} --exec-server-task=${exec-task} -T typed ${root} $(task-create) $(task-resume)',\ + > 'exec.static $(exec-task=task-create)' Note that, contrary to [[GRUB]]'s configuration file, you don't specify "`argv[0]`" here, and it's fortunate that neither ext2fs nor exec need a comma on their command line... -You can also use `--append [...]`, which will show up in `/proc/cmdline`. - -Command line above crashes with old qemu versions, for instance qemu 1.1.2 on Debian Wheezy, fixed by upgrading to wheezy-backports currently qemu 1.7.0, see [[!debbug 741873]] - --- # Related Links @@ -394,4 +443,4 @@ system after installation. [[Image_for_L4]] -- a QEMU image for the Hurd/L4 project. -<http://eyeside.net/hurd/Hurd-on-QEMU.html> +[Hurd Installation Guide from eyeside.net](https://web.archive.org/web/20130516025306/http://www.eyeside.net/index.htm) diff --git a/hurd/subhurd.mdwn b/hurd/subhurd.mdwn index a92a8d3f..d24369bc 100644 --- a/hurd/subhurd.mdwn +++ b/hurd/subhurd.mdwn @@ -37,6 +37,11 @@ boot it: $ gunzip debian-hurd.img.gz $ boot --kernel-command-line="fastboot root=pseudo-root" -T typed part:1:file:debian-hurd.img +/!\ If you face an error from the mach-defpager (most probably +because there is already a default pager), you can comment +the part that says `/hurd/mach-defpager` from the `/etc/hurd/runsystem.sysv` file +included within the `debian-hurd.img` file you are trying to use. + The 'fastboot' is necessary to skip the filesystem check which fails because the image assumes the root filesystem to be /etc/hd0s1. Once booted, you can correct this: @@ -77,7 +82,7 @@ debootstrap as root: mke2fs /dev/hd0s6 settrans -ca mnt /hurd/ext2fs /dev/hd0s6 - debootstrap sid mnt/ http://httpredir.debian.org/debian + debootstrap --keyring=/usr/share/keyrings/debian-ports-archive-keyring.gpg --extra-suites=unreleased sid chroot http://deb.debian.org/debian-ports/ settrans -fga mnt ## Booting @@ -134,7 +139,7 @@ In the subhurd, you can do basically all the same things as in the main Hurd. You can even set up networking: Just invoke `settrans` on the `/servers/socket/2` as usual inside the subhurd, using `/dev/eth0`, only using a different local IP than in the main Hurd. This way, the subhurd will be able to communicate to -the outside world with its own IP -- allowing for example to do `apt-get` +the outside world with its own IP -- allowing for example to do `apt inside the subhurd, or to `ssh` directly into the subhurd. If you want to access the subhurd processes from the outside, e.g. for diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index 32562a8b..dad26881 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -92,6 +92,7 @@ The [[concept|concepts]] of translators creates its own problems, too: * [[exec]] * [[proc]] * [[pfinet]] +* [[lwip]] * [[eth-filter]] * [[pflocal]] * [[hostmux]] @@ -107,6 +108,7 @@ The [[concept|concepts]] of translators creates its own problems, too: * [[firmlink]] * [[fifo]] * [[term]] +* [[checkperms]] * ... diff --git a/hurd/translator/checkperms.mdwn b/hurd/translator/checkperms.mdwn new file mode 100644 index 00000000..a8a52cb1 --- /dev/null +++ b/hurd/translator/checkperms.mdwn @@ -0,0 +1,233 @@ +[[!meta copyright="Copyright © 2021 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The *checkperms* translator implements deferred authorization. + +It is part of a project to enable asking for a grant of authorization +when processes access a file. It is built as a translator and a simple +permission granting program. + +The translator can delegate permission-granting to the program via two +FIFO files. The goal is to create a simple replacement for the +use-case of polkit of granting privilege to a process to access some +resource after user-interaction with a permission-granting daemon. + + +# Code + +The translator is available in the checkperm-deferred-authorization branch in [the hurd repository](https://git.savannah.gnu.org/cgit/hurd/hurd.git). + +The code for the program is provided in this article + +# Usage Example + +We restrict a the node /hello to require explicit permission for every +PID that does not have the group `user`. This notably does include +processes started by root. + + +## How it looks + +**First shell** as root: + + settrans -cga /hello $(realpath ~/Dev/hurd/trans/checkperms) --groupname=user + su - user --shell /bin/bash -c 'cat /hello' + # ⇒ HELLOWORLD # user has the group user + cat /hello # root does not have the group user, so + # this blocks until positive reply in the other shell + +**Second shell** (run the program): + + Process 732 tries to access file /hello but is not in the required group user. + USER PID %CPU %MEM SZ RSS TT STAT START TIME COMMAND + root 732 0.0 0.1 148M 3.55M p2 Sso Mon 1AM 0:01.10 -bash + Grant permission and add group "user" for 5 minutes? [y/N]> y + +**First shell** as root: + + # ⇒ HELLOWORLD + # only blocks once despite getting two reads from cat, + # because for the second read cat already has the group `user`. + + + +## Trying it yourself + +Setup the development environment with the code at ~/Dev similar to +https://www.draketo.de/software/hurd-development-environment + + +Compile and setup the translator: + + cd ~/Dev/hurd && \ + patch -p1 < checkperms.patch && \ + autoreconf -i && \ + ./configure --without-parted && \ + make && \ + touch trans/checkperms.c && \ + CFLAGS="$CFLAGS -g" make && \ + echo HELLOWORLD > /hello && \ + settrans -cga /hello $(realpath ~/Dev/hurd/trans/checkperms) --groupname=user + +Create the FIFOs: + + USER=root + GROUP=user + mkdir -p /run/$USER/request-permission + mkdir -p /run/$USER/grant-permission + mkfifo /run/$USER/request-permission/$GROUP + mkfifo /run/$USER/grant-permission/$GROUP + +Setup the permission-granting program in a separate shell: + + USER=root + GROUP=user + while true; do + PID="$(cat /run/$USER/request-permission/$GROUP)" + echo Process $PID tries to access file /hello but is not in the required group $GROUP. + ps-hurd -p $PID -aeux + if [[ "$(read -e -p 'Grant permission and add group "'$GROUP'" for 5 minutes? [y/N]> '; echo $REPLY)" == [Yy]* ]]; then + addauth -p $PID -g $GROUP + echo 0 > /run/$USER/grant-permission/$GROUP + (sleep 300 && rmauth -p $PID -g $GROUP 2>/dev/null) & + else + echo 1 > /run/$USER/grant-permission/$GROUP + fi + done + + +Access the translator as user without the required group and with the group: + + su - user --shell /bin/bash -c cat /hello' + cat /hello & + + +# Concept + +## The translator + +The translator is started with a GROUP as argument. When the file is +accessed, the translator checks whether the process has the given +group. If it does, it returns data read from the underlying file. + +If the process lacks the required group, the translator retrieves its +USER and PID and writes the PID into a FIFO located at + + /run/USER/request-permission/GROUP + +Then it reads from + + /run/USER/grant-permission/GROUP + +It blocks until it gets a reply. If it reads a 0 (=success), it reads +from the file and returns the data. + +## The permission granting program + +The permission granting program reads the PID from + + /run/USER/request-permission/GROUP + +retrieves information about the PID and asks the user whether to allow +the program. + +If the USER answers no, the RET value is non-zero. + +If the USER answers yes, the RET value is zero (0) +and the program adds the GROUP to the process at PID (using addauth). + +It also starts a daemon that will remove the group again after 5 +minutes (modelled after the temporary permissions to run privileged +without password granted by sudo). + +The program then writes the RET value into + + /run/USER/grant-permission/GROUP + +## What if the translator crashes? + +If the translator crashes, the permissions return to those of the +underlying node. For every user except root this usually means that +the process does not have access to the file. + +The failure-mode should therefore be safe. + +# Possibilities + +The most important use-case for this translator is to make it easier +to start programs with reduced permissions and only add these when +required. + +To setup deferred permissions for a single file, you can create a +group just for that file. Then each file can have its own permission +granting program. Having dedicated groups decouples authentication and +authorization while staying in the conventional *nix permissions +scheme. + +You can also set this translator on a file that gets accessed first +when a process accesses a set of related files that all have the same +group. Since the authorization-program here adds the group for 5 +minutes, the other files can afterwards be accessed, too. + +Since the translator simply defers to a program, that program could do +any action to get authorization, including `curl`. Administrators for +a local network could therefore set up terminals for unprivileged +users that request permissions from a local server when accessing a +file. That way permissions can easily be coordinated over multiple +machines. (naturally this does not restrict root who can always use +settrans -g to get raw access to the file) + + + + +# Open Issues + +## read-only + +[[!tag open_issue_hurd]] + +The current implementation only provides read-access, writing is +prevented. This is not an intrinsic limitation, only an implementation +artefact. + +## delegate + +The underlying file is currently read by the translator and the data +returned to the reading process. To reduce delays, it could directly +delegate to the underlying file. With the long term goal to provide +multiplexing of access, for example for audio, reading via the +translator could be preferable, though. + +## writing via system shell + +Writing to and reading from the FIFOs is currently done with +`system()`. It would be nicer to move to an implementation that does +not rely on the system-shell. + +## potential race-condition + +Accesses from two different translators can currently race for the +reply. To fix this, the translator should write the PID and a random +LABEL into the request. The program should repeat that label for +replies to ensure that the reply and request can be matched. If +receiving a non-matching reply, it MUST be written into the grant +again after a random delay to enable a matching translator to +retrieve the grant. +REQUEST: PID LABEL +GRANT: RET LABEL (RET=0 is success) +LABEL=$RANDOM + + +## multiple permission-granting programs + +The system assumes having a single permission granting program per +user. For a setup with multiple unconnected sessions per user (like +several TTYs) the permission granting program needs to coordinate +between these. diff --git a/hurd/translator/httpfs.mdwn b/hurd/translator/httpfs.mdwn index 8b02aa06..0ce0f30b 100644 --- a/hurd/translator/httpfs.mdwn +++ b/hurd/translator/httpfs.mdwn @@ -12,6 +12,84 @@ License|/fdl]]."]]"""]] While the httpfs translator works, it is only suitable for very simple use cases: it just provides the actual file contents downloaded from the URL, but no additional status information that are necessary for interactive use. (Progress indication, error codes, HTTP redirects etc.) +# Introduction + +Here we describe the structure of the /http filesystem for the Hurd. +Under the Hurd, we provide a translator called 'httpfs' which is intended +to provide the filesystem structure. + +The httpfs translator accepts an "http:// URL" as an argument. The underlying +node of the translator can be a file or directory. This is guided by the --mode +command lineoption. Default is a directory. + +If its a file, only file system read requests are supported on that node. If +its a directory, we can cd into that directory and ls would list the files in +the web server. A web server may provide a directory listing or it may not +provide, whatever it be the case the web server always returns an HTML stream +for an user request (GET command). So to get the files residing in the web +server, we have to parse the incoming HTML stream to find out the anchor +tags. These anchor tags point to different pages or files in the web +server. These file name are extracted and filled into the node of the +translator. An anchor tag can also be a pointer to an external URL, in such a +case we just show that URL as a regular file so that the user can make file +system read requests on that URL. In case the file is a URL, we change the name +of URL by converting all the /'s with .'s so that it can be displayed in the +file system. + +Only the root node is filled when the translator is set, subdirectories inside +that are filled as on demand, i.e. when a cd or ls occurs on that particular sub +directory. + +The File size is now displayed as 0. One way of getting individual file sizes is +sending a GET request for each file and cull the file size from Content-Length +field of an HTTP response. But this may put a very heavy burden on the network, +So as of now we have not incorporated this method with this http translator. + +The translator uses the libxml2 library for doing the parsing of HTML +stream. The libxml2 provides SAX interfaces for the parser which are used for +finding the begining of anchor tags `<A href="i.html">`. So the translator has +dependency on the libxml2 library. + +If the connection to the Internet through a proxy, then the user must explicitly +give the IP address and port of the proxy server by using the command line +options --proxy and --port. + + +# How to Use httpfs + + # settrans -a tmp/ /hurd/httpfs http://www.gnu.org/software/hurd/index.html + +<Remember to give the / at the end of the URL, unless you are specifying a specific file like www.hurd-project.com/httpfs.html > + + # cd tmp/ + + # ls -l + + # settrans -a tmp/ /hurd/httpfs http://www.gnu.org/software/hurd/index.html --proxy=192.168.1.103 + --port=3126 + +The above command should be used in case if the access to the Internet is +through a proxy server, substitute your proxies IP and port no.s + +# TODO + +- https:// support +- scheme-relative URL support (eg. "//example.com/") +- query-string and fragment support +- HTTP/1.1 support +- HTTP/2 support +- HTTP/3 support (there may exist a C library that provides HTTP/[123] + support). +- Teach httpfs to understand HTTP status codes like re-directs, 404 not found, + etc. +- Teach httpfs to look for "sitemaps". Many sites offer a sitemap, and this + would be a nifty way for httpfs to allow grep-ing the entire site's + contents. [[sitemaps.org|https://www.sitemaps.org]] is a great resource for + this. +- Teach httpfs to check if the computer has an internet connection at + startup and during operation. The translator causes 30 second + pauses on commands like "ls", when the internet is down. + # Source <http://www.nongnu.org/hurdextras/#httpfs> diff --git a/hurd/translator/lwip.mdwn b/hurd/translator/lwip.mdwn index efa59285..fab7d6f2 100644 --- a/hurd/translator/lwip.mdwn +++ b/hurd/translator/lwip.mdwn @@ -16,7 +16,10 @@ To configure lwip for internet connectivity, use the The argument /server/socket/2 is the node that the translator is to be attached to. This is followed by the translator program to run and any arguments to give it. -There, -i, -a, -g and -m are, quite obviously, the (Mach) device to use, the IP address, the gateway and netmask. +There, -i, -a, -g and -m are, quite obviously, the (Mach) device to use, the IP +address, the gateway and netmask. You can discover these values via the +`ifconfig` command (You need to run this command on the host system and NOT in +the qemu environment). More information can be found on Joan Lledo's blog: diff --git a/hurd/translator/pfinet/ipv6.mdwn b/hurd/translator/pfinet/ipv6.mdwn index ccb359cb..d864e256 100644 --- a/hurd/translator/pfinet/ipv6.mdwn +++ b/hurd/translator/pfinet/ipv6.mdwn @@ -139,7 +139,7 @@ Indeed, IPv6 now works properly, and the very machine hosting this wiki <youpi> which repo? <youpi> I don't have such commit here <gnu_srs> - http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=2b2d7fdc42475019e5ce3eabc9c9673e3c13d89f + https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=2b2d7fdc42475019e5ce3eabc9c9673e3c13d89f <gnu_srs> From which release, 2.4.x, 2.6.x? <youpi> it's very old <youpi> 2002 diff --git a/hurd/translator/procfs.mdwn b/hurd/translator/procfs.mdwn index 0228d4d4..b3753592 100644 --- a/hurd/translator/procfs.mdwn +++ b/hurd/translator/procfs.mdwn @@ -31,7 +31,7 @@ Madhusudan.C.S has implemented a new, fully functional [[procfs|madhusudancs]] f In August 2010, Jérémie Koenig [published another, new version](http://lists.gnu.org/archive/html/bug-hurd/2010-08/msg00165.html). -This can be found in <http://git.savannah.gnu.org/cgit/hurd/procfs.git/>. +This can be found in <https://git.savannah.gnu.org/cgit/hurd/procfs.git/>. Testing it is as simple as this: diff --git a/hurd/translator/tmpfs.mdwn b/hurd/translator/tmpfs.mdwn index 3d5cb74e..4db6542b 100644 --- a/hurd/translator/tmpfs.mdwn +++ b/hurd/translator/tmpfs.mdwn @@ -20,6 +20,18 @@ system|ext2fs]] on it, having a real `tmpfs` is better, as it need not deal with the additional block-level indirection layer that `ext2` (or any other disk-based file system) imposes. -`tmpfs` generally works, although it requires root permissions for file content; -see the [[discussion]] sub-pages for the past and current issues. -There is a [[!FF_project 271]][[!tag bounty]] on this task. +`tmpfs` generally works. See the [[discussion]] sub-pages for the +past and current issues. There is a [[!FF_project 271]][[!tag +bounty]] on this task. + +## How to use tmpfs + + $ settrans -ac tmp /hurd/tmpfs 1MB + $ cd tmp + $ touch file + $ cat file + + $ echo "tmpfs rocks!" > ./file + $ cat file + tmpfs rocks! + $
\ No newline at end of file diff --git a/hurd/translator/tmpfs/discussion.mdwn b/hurd/translator/tmpfs/discussion.mdwn index 72400121..d61fd796 100644 --- a/hurd/translator/tmpfs/discussion.mdwn +++ b/hurd/translator/tmpfs/discussion.mdwn @@ -107,7 +107,7 @@ License|/fdl]]."]]"""]] <antrik> mcsim: did you publish your in-progress work? <mcsim> there is a branch with working tmpfs in git repository: - http://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=mplaneta/tmpfs/defpager + https://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=mplaneta/tmpfs/defpager <jd823592> sorry for interrupting the meeting but i wonder what is a lazyfs? <mcsim> jd823592: lazyfs is tmpfs which uses own pager diff --git a/hurd/translator/ufs.mdwn b/hurd/translator/ufs.mdwn index 4d611e95..9e9c6f75 100644 --- a/hurd/translator/ufs.mdwn +++ b/hurd/translator/ufs.mdwn @@ -19,7 +19,7 @@ and will eat your data. <Arne`> There might be a copyright problem: <nalaginrut> well, there seems BSD-4clauses in the code: - http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/ufs/alloc.c + https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/ufs/alloc.c <Arne`> braunr, tschwinge: Do you have any info on that? 4-clause BSD and GPL on the same code are a license incompatibility… <tschwinge> Arne`: I've put it onto my (long) TODO list. diff --git a/hurd/translator/unionfs.mdwn b/hurd/translator/unionfs.mdwn index 06524f3e..31162c37 100644 --- a/hurd/translator/unionfs.mdwn +++ b/hurd/translator/unionfs.mdwn @@ -15,7 +15,7 @@ License|/fdl]]."]]"""]] *Unionfs allows you to simply union one directory or translator into another one, so you see the files of both of them side by side.* -Source repository: <http://git.savannah.gnu.org/cgit/hurd/unionfs.git/> +Source repository: <https://git.savannah.gnu.org/cgit/hurd/unionfs.git/> Right now there are some problems with syncing, so please be aware that it might not work as expected. diff --git a/hurd/translator/xmlfs.mdwn b/hurd/translator/xmlfs.mdwn index a4de1668..6028d43f 100644 --- a/hurd/translator/xmlfs.mdwn +++ b/hurd/translator/xmlfs.mdwn @@ -11,6 +11,80 @@ License|/fdl]]."]]"""]] `xmlfs` is a translator that provides access to XML documents through the filesystem. +# How to Use xmlfs + + xmlfs - a translator for accessing XML documents + +This is only an alpha version. It works in read only. It supports +text nodes and attributes. It doesn't do anything fancy like size +computing, though. Here is an example of how to use it: + + $ wget http://cvs.savannah.nongnu.org/viewvc/*checkout*/hurdextras/xmlfs/example.xml?content-type=text%2Fplain; + $ settrans -ca xml /hurd/xmlfs example.xml #the website says to use ./xmlfs + $ cd xml; ls + library0 library1 + $ cd library0; ls -A + .text1 .text2 @name book0 book1 book2 sub-library0 sub-library1 + $ cat .text2 + +CDATA, again ! + + $ cat book0 + <book> + <author>Mark Twain</author> + <title>La case de l'oncle Tom</title> + <isbn>4242</isbn> + </book> + $ cat book0/author/.text + Mark Twain + +As you can see, text nodes are named .textN, with N an integer +starting from 0. Sorting is supposed to be stable, so you get the same +N every time you access the same file. If there is only one text node +at this level, N is ommitted. Attributes are prefixed with @. + +An example file, example.xml, is provided. Of course, it does not +contain anything useful. xmlfs has been tested on several-megabytes +XML documents, though. + +Comments are welcome. + + -- Manuel Menal <mmenal@hurdfr.org> + +# TODO +- Handle memory usage in a clever way: + - do not dump the nodes at each read, try to guess if read() + is called in a sequence of read() operations (e.g. cat reads + 8192 bytes by 8192 bytes) and if it is, cache the node + contents. That'd need a very small ftpfs-like GC. + - perhaps we shouldn't store the node informations from + first access to end and have a pool of them. That might come + with next entries though. +- Handle changes of the backing store (XML document) while running. + (Idea: we should probably attach to the XML node and handle + read()/write() operations ourselves, with libxml primitives.) +- Write support. Making things like echo >, sed and so on work is + quite obvious. Editing is not -that- simple, 'cause we could + want to save a not XML well-formed, and libxml will just return + an error. Perhaps we should use something like 'sync'. +- Handle error cases in a more clever way ; there are many error + conditions that will just cause xmlfs to crash or do strange + things. We should review them. +- Make sorting *really* stable. + +# TODO WISHLIST +-------- + +- Kilobug suggested a --xslt option that would make xmlfs provide + a tree matching the XSLT-modified document. + (Problem: In this case we cannot attach easily to the .xml 'cause + the user would loose access to theirs original document. Perhaps + we should allow an optional "file.xml" argument and check if it + is not the same as the file we are attaching to when --xslt is + specified.) +- DTD support ; perhaps XML schema/RelaxNG when I'm sure I understand + them ;-) + # Source |