diff options
Diffstat (limited to 'open_issues')
29 files changed, 2329 insertions, 1097 deletions
diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn index b3e826b7..b85d95de 100644 --- a/open_issues/64-bit_port.mdwn +++ b/open_issues/64-bit_port.mdwn @@ -19,11 +19,21 @@ License|/fdl]]."]]"""]] **For pure 64bit support, we need to** - * bootstrap a distrib + * bootstrap a distribution. Debian is done. Alpine-Hurd is mostly + bootstrapped. Reach out to Sergey if you are interested in an + Alpine-Hurd distribution with bleeding-edge software. Reach out + to Guix, if you want to help them bootstrap a 64bit Hurd. * port gdb * Fix bugs :) * Notably it seems to be requiring at least 2G memory to boot. +**Installing via the installer image** + +The installer appears to work in qemu. Give it a shot! Let us know +how it works in real hardware! + +[[http://cdimage.debian.org/cdimage/ports/latest/hurd-amd64/current/]] + **Installing a 64bit chroot** You can use the pre-built image from https://people.debian.org/~sthibault/hurd-i386/initrd-amd64.img.gz and boot that. @@ -34,7 +44,11 @@ Make sure to have `debootstrap >= 1.0.128+nmu2+hurd.1` mkdir chroot-hurd-amd64/etc/apt/trusted.gpg.d ln -s /usr/share/keyrings/debian-keyring.gpg chroot-hurd-amd64/etc/apt/trusted.gpg.d/ -Then boot it, it will drop you into a shell. You need to make / writable: +Then boot it, it will drop you into a shell. You'll probably want to use a nicer shell: + + bash + +You need to make / writable: fsysopts / --writable @@ -66,10 +80,10 @@ After reboot, you'll probably want to setup network: vi /etc/network/interfaces # put there this: - # auto /dev/eth0 - # iface /dev/eth0 inet static - # address 10.0.2.15/16 - # gateway 10.0.2.2 + auto /dev/eth0 + iface /dev/eth0 inet static + address 10.0.2.15/16 + gateway 10.0.2.2 **Creating a 64bit disk image** @@ -82,7 +96,7 @@ To make a bootable system we really better make the disk image partitioned, and # create a new primary partition spanning the whole disk: n p and just accept the defaults, and finish with w settrans -ca disk /hurd/storeio -T typed file:disk.img settrans -ca disk1 /hurd/storeio -T typed part:1:file:disk.img - mke2fs disk1 + /sbin/mke2fs -E root_owner=$UID:0 disk1 settrans -ca chroot-hurd-amd64 /hurd/ext2fs disk1 (here we assume that fdisk puts the partition at sector 2048, that's indeed the @@ -119,16 +133,8 @@ You can then make the disk bootable: settrans -ga disk settrans -ga disk1 -Note: you need to set the uuid in the grub config according to the uuid produced by the `mke2fs` command. - Then boot it, and proceed like for the chroot case. -**Installing from the debian-ports archive** - -For now it's quite empty (not even gcc), but it can be debootstrapped. That will be used to build packages on the buildds. - - debootstrap --foreign --verbose --arch hurd-amd64 --extra-suites=unreleased --include=debian-ports-archive-keyring --keyring=/usr/share/keyrings/debian-ports-archive-keyring.gpg sid chroot-hurd-amd64 https://deb.debian.org/debian-ports/ - **Creating a pbuilder chroot** Here is a sample `/etc/pbuilderrc`: @@ -155,6 +161,19 @@ And this is needed until we get the `aptitude` package built: And then you can run `sudo pbuilder create` , `sudo pbuilder login` , `pdebuild` +**Installing from the debian-ports archive** + +For now it's quite empty (not even gcc), but it can be debootstrapped. That will be used to build packages on the buildds. + + debootstrap --foreign --verbose --arch hurd-amd64 --extra-suites=unreleased --include=debian-ports-archive-keyring --keyring=/usr/share/keyrings/debian-ports-archive-keyring.gpg sid chroot-hurd-amd64 https://deb.debian.org/debian-ports/ + +**Installing proper & more packages** + +The `people.debian.org` repository is only for bootstraping the distribution. Proper packages are getting uploaded on the usual mirror: + + deb http://deb.debian.org/debian-ports sid main + deb http://deb.debian.org/debian-ports unreleased main + **Installing a 64bit system** In principle crosshurd should be working, one however should add this source to get more packages for now: diff --git a/open_issues/_san.mdwn b/open_issues/_san.mdwn index 5980784b..374e89fe 100644 --- a/open_issues/_san.mdwn +++ b/open_issues/_san.mdwn @@ -35,10 +35,16 @@ as: * Undefined Behavior Sanitizer (UBsan; `-fsanitize=undefined`) Porting these to the Hurd is not a trivial task, for they have intimate -knowledge about the operating system kernel they're running on, and from a -first look they reimplement a lot of [[/glibc]] by directly using +knowledge about the operating system kernel they're running on, and +reimplement the needed parts of [[/glibc]] by directly using [[system_call]]s -- which is basically a no-go on GNU Hurd. +Samuel took some look at it and got some basic support for libubsan and libasan +by making it call glibc still: + <https://people.debian.org/~sthibault/hurd-i386/libubsan-hurd.diff> +which allowed to fix some issues in the hurd code. + +This needs to be updated to a newer gcc and submitted upstream. # IRC, OFTC, #gcc, 2012-12-11 diff --git a/open_issues/alarm_setitimer.mdwn b/open_issues/alarm_setitimer.mdwn index a1c8a7d3..5f121ca6 100644 --- a/open_issues/alarm_setitimer.mdwn +++ b/open_issues/alarm_setitimer.mdwn @@ -58,7 +58,7 @@ This issue was recently fixed (around January 2013). (e19a2fad70b187e5efe79768f86a1f05cb5e0390, Tue Feb 21 02:41:18 2012) <braunr> yes, fixed :) <braunr> patch committed at - http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/setitimer_fix + https://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/setitimer_fix <youpi> and pushed to the debian package diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 82f2333c..ed1aaf97 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -32,6 +32,9 @@ depend on GNU Mach, while other Hurd servers depend on other Hurd servers. # Bootstrap +Take a look at the [[hurd/bootstrap]] page. There is an alternative +proposal for an [[different bootstrap|open_issues/serverbootv2]]. + ## [[hurd_init]] ## Hurd Booting Process @@ -1050,12 +1053,12 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <teythoon> as I said, the wire format is specified, the semantics only in written form in the source <teythoon> this is an example of the ipc specification for the proc server - http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/hurd/process.defs + https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/hurd/process.defs <crocket> teythoon, how file server interacts with file clients should be defined as a formal protocol, too. <teythoon> do you consider the ipc description a kind of formal protocol ? <crocket> - http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/hurd/process.defs can + https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/hurd/process.defs can be considered as a formal protocol. <crocket> However, the file server protocol should be defined on top of IPC protocol. diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn index 26e0b770..cc33b074 100644 --- a/open_issues/arm_port.mdwn +++ b/open_issues/arm_port.mdwn @@ -9,57 +9,444 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -Several people have expressed interested in a port of GNU/Hurd for the ARM -architecture. +An experimental AArch64 port of GNU Mach, the Hurd servers, and glibc +is underway. In late September of 2024 the Hurd developers were able +to run a minimal Hurd environment in qemu. It still needs lots of +work before we reccommend it to the casual user. +Typically when one ports the Hurd to a different CPU architecture, one +needs to first port GNU Mach. But Sergey actually did it backward. +He ported the Hurd first! He did some hacking on glibc, binutils, +GCC, and added some headers to GNU Mach. Then He was able to build +the core Hurd servers: ext2fs, proc, exec, and auth. -# IRC, freenode, #hurd, 2012-07-28 +One would think that he would need to port GNU Mach to run the +binaries, but Sergey ran a statically linked hello world executable on +GNU/Linux, under GDB, being careful to skip over and emulate syscalls +and RPCs. The glibc port has the TLS setup, hwcaps / cpu-features, +and ifuncs. Then he went and ported GNU Mach with help from Luca. +Once Sergey's glibc patches are merged, then we can start cross +compiling debian packages for aarch64-gnu. - <mcsim> Has anyone heard about porting hurd and gnu/mach to arm - architecture? - <braunr> mcsim: i think so - <braunr> mcsim: why are you asking ? - <mcsim> I found an article where author stated that he has ported hurd to - arm, but I have never met this information before. - <mcsim> He wrote ethernet driver and managed to use ping command - <mcsim> author's name is Sartakov Vasily - <braunr> well that's possible, a long time ago - <braunr> and it was probably not complete enough to be merged upstream - <braunr> like many other attempts at many other things - <mcsim> Not so long. Article is dated by June 2011. - <braunr> do you have a link ? - <mcsim> Yes, but it is in Russian. - <braunr> oh - <braunr> well i don't remember him sharing that with us - <antrik> mcsim: he did some work on porting Mach, but AIUI never got it - nearly finished - <antrik> nowadays he does L4 stuff - <antrik> was also at FOSDEM +azert on irc can run a minimal GNU Mach userspace on the Olimex A64 +OLinuXino board, which supports a max 2GB of RAM. As of September of +2024, the easiest way to develop on the AArch64 port, is with qemu. +Keep reading and you will see the guide that somes you how to +cross-build a AArch64 Hurd. +Now to some of the more technical things. -## IRC, freenode, #hurd, 2012-10-09 +- The TLS implementation is basically complete and working. We're +using `tpidr_el0` for the thread pointer (as can be seen in the listing +above), like GNU/Linux and unlike Windows (which uses x18, apparently) +and macOS (which uses `tpidrro_el0`). We're using "Variant I" layout, as +described in "ELF Handling for Thread-Local Storage", again same as +GNU/Linux, and unlike what we do on both x86 targets. This actually +ends up being simpler than what we had for x86! The other cool thing +is that we can do `msr tpidr_el0, x0` from userspace without any +gnumach involvement, so that part of the implementation is quite a bit +simpler too. - <mcsim> bootinfdsds: There was an unfinished port to arm, if you're - interested. - <tschwinge> mcsim: Has that ever been published? - <mcsim> tschwinge: I don't think so. But I have an email of that person and - I think that this could be discussed with him. +- Conversely, while on x86 it is possible to perform "cpuid" and +identify CPU features entirely in user space, on AArch64 this requires +access to some EL1-only registers. On Linux and the BSDs, the kernel +exposes info about the CPU features via `AT_HWCAP` (and more recently, +`AT_HWCAP2`) auxval entries. Moreover, Linux allows userland to read +some otherwise EL1-only registers (notably for us, `midr_el1`) by +catching the trap that results from the EL0 code trying to do that, +and emulating its effect. Also, Linux exposes `midr_el1` and +`revidr_el1` values through procfs. +- The Hurd does not use `auxval`, nor is gnumach involved in `execve` anyway. +So I thought the natural way to expose this info would be with an RPC, +and so in `mach_aarch64.defs` I have an `aarch64_get_hwcaps` routine that +returns the two hwcaps values (using the same bits as `AT_HWCAP{,2}`) and +the values of `midr_el1`/`revidr_el1`. This is hooked to `init_cpu_features` +in glibc, and used to initialize `GLRO(dl_hwcap)` / `GLRO(dl_hwcap2)` and +eventually to pick the appropriate ifunc implementations. -## IRC, freenode, #hurd, 2012-10-10 +- The page size (or rather, paging granularity) is notoriously not +necessarily 4096 on ARM, and the best practice is for userland not to +assume any specific page size and always query it dynamically. GNU +Mach will (probably) have to be built support for some specific page +size, but I've cleaned up a few places in glibc where things were +relying on a statically defined page size. - <tschwinge> mcsim: If you have a contact to the ARM porter, could you - please ask him to post what he has? - <antrik> tschwinge: we all have the "contact" -- let me remind you that he - posted his questions to the list... +- There are a number of hardware hardening features available on AArch64 +(PAC, BTI, MTE — why do people keep adding more and more workarounds, +including hardware ones, instead of rewriting software in a properly +memory-safe language...). Those are not really supported right now; all +of them would require some support form gnumach side; we'll probably +need new protection flags (`VM_PROT_BTI`, `VM_PROT_MTE`), for one thing. +We would need to come up with a design for how we want these to work +Hurd-wide. For example I imagine it's the userland that will be +generating PAC keys (and settings them for a newly exec'ed task), +since gnumach does not contain the functionality to generate random +values (nor should it); but this leaves an open question of what +should happen to the early bootstrap tasks and whether they can start +using PAC after initial startup. -## IRC, freenode, #hurd, 2012-10-17 +- Unlike on x86, I believe it is not possible to fully restore +execution context (the values of all registers, including `pc` and +`cpsr`) purely in userland; one of the reasons for that being that we +can apparently no longer do a load from memory straight into `pc`, +like it was possible in previous ARM revisions. So the way `sigreturn +()` works on Linux is of course they have it as a syscall that takes a +`struct sigcontext`, and writes it over the saved thread state, which +is similiar to `thread_set_state ()` in Mach-speak. The difference +being that `thread_set_state ()` explicitly disallows you to set the +calling thread's state, which makes it impossible to use for +implementing `sigreturn ()`. So I'm thinking we should lift that +restriction; there's no reason why `thread_set_state ()` cannot be +made to work on the calling thread; it only requires some careful +coding to make sure the return register (`%eax`/`%rax`/`x0`) is *not* +rewritten with `mach_msg_trap`'s return code, unlike normally. - <mcsim> tschwinge: Hello. The person who I wrote regarding arm port of - gnumach still hasn't answered. And I don't think that he is going to - answer. +- We need an interrupt handling framework with interupt controller + support. Jessica Clark apparently did some of this for FreeBSD. + Perhaps Jessica could give us some pointers. +But other than that, I do have an AArch64 versions of `trampoline.c` +and `intr-msg.h` (complete with `SYSCALL_EXAMINE` & +`MSG_EXAMINE`). Whether they work, we'll only learn once we have +enough of the Hurd running to have the proc server. + +MIG seems to just work (thanks to all the Flávio's work!). We are +using the `x86_64` ABI, and I have not seen any issues so far — +neither compiler errors / failed static assertions (about struct sizes +and such), nor hardware errors from misaligned accesses. + +To bootstrap gnumach someone must fix the console, set up the virtual +memory, thread states, context switches, irqs and userspace entry +points, etc. + +Also, there is a bunch of design work to do. + +Will/can AArch64 use the same mechanism for letting userland handle +interrupts? Do we have all the mechanisms required for userland to +poke at specific addresses in memory (to replace I/O ports)? — I +believe we do, but I haven't looked closely. + +AFAIK there are no I/O ports in ARM, the usual way to configure things +is with memory-mapped registers, so this might be easy. About IRQs, +probably it needs to be arch-specific anyway. + +What should the API for manipulating PAC keys look like? Perhaps it +should be another flavor of thread state, but then it is really +supposed to be per-task, not per-thread. Alternatively, we could add a +few aarch64-specific RPCs in `mach_arrch64.defs` to read and write the +PAC keys. But also AFAICS Mach currently has no notion of per-task +arch-specific data (unlike for threads, and other than the VM map), so +it'd be interesting to add one. Could it be useful for something +else? + +What are the debugging facilities available on ARM / AArch64? Should +we expose them as more flavors of thread state, or something else? +What would GDB need? + +Should gnumach accept tagged addresses (like `PR_SET_TAGGED_ADDR_CTRL` +on Linux)? + +Can we make Linux code (in-Mach drivers, pfinet, netdde, ...) work on +AArch64? + +One can trivially port pfinet to AArch64. Eventually, we should fix +any remaining issues with lwip. That way we can stop spending time +maintaining pfinet, which is Linux's old abandoned networking stack. + +Developers will have a difficult time porting the in-Mach drivers +(arm64 was probably not even a thing at the time). We can perhaps +port Netdde, but we should instead get our userspace drivers from a +rumpkernel. + +Starting the kernel itself should be easy, thanks to GRUB, but it +shouldn't be too hard to add support for U-Boot either if needed. + +I think more issues might come out setting up the various pieces of +the system. For example, some chips have heterogeneous cores, +(e.g. mine has two A72 cores and four A53 cores) so SMP will be more +complicated. + +Also, about the serial console, it might be useful at some point to +use a driver from userspace, if we can reuse some drivers from netbsd +or linux, to avoid embedding all of them in gnumach. + +# Setting up a AArch64 development environment (in qemu) + +Before you follow the below guide, take a look at Flavio's +[[cross-hurd|https://github.com/flavioc/cross-hurd]] project, which +should give you some ideas about how to [[cross +build|toolchain/cross-gnu]] the AArch64 GNU Mach and the Hurd via a +GNU/Linux device. You can also read Sergey's +[[guide|https://mail.gnu.org/archive/html/bug-hurd/2023-01/msg00132.html]] +on how to cross build a X86 Hurd (but you will need a more recent +binutils and gcc). + +## Build gnumach + +Prepare the crosscompiler environment + + $ PREFIX=~/src/crosshurd-aarch64 + $ mkdir $PREFIX + $ mkdir $PREFIX/{src,include,aarch64-gnu}; ln -s ../include + $ $PREFIX/aarch64-gnu/sys-include; ln -s ../lib $PREFIX/aarch64-gnu/lib + +Compile binutils + + $ cd $PREFIX/src + $ git clone https://sourceware.org/git/binutils-gdb.git + $ mv binutils-gdb/ binutils + $ cd binutils + $ mkdir build + $ cd build + $ ../configure --disable-werror --target=aarch64-gnu --prefix=$PREFIX --with-lib-path=$PREFIX/lib + $ make + $ make install + +Verify that e.g. `$PREFIX/bin/i686-gnu-nm -v` works. + +Compile gcc + + $ cd $PREFIX/src + $ git clone https://gcc.gnu.org/git/gcc.git + $ cd gcc + $ mkdir build + $ cd build + $ ../configure --prefix=$PREFIX --target=aarch64-gnu \ + --with-lib-path=$PREFIX/lib --disable-multilib --enable-languages=c + $ make -j2 all-gcc + $ make install-gcc + +Note that at this step we're only building GCC itself, not libgcc. +We can build libgcc after we have installed glibc headers. + +Next, we want to build MIG. MIG needs Mach headers, but we can't build +GNU Mach without MIG. So what we do is we 'make' GNU Mach twice, once +to install the headers, and later on once more to actually build it. + +You might need this +[[patch|https://lists.gnu.org/archive/html/bug-hurd/2024-03/msg00110.html]]. + +Compile gnumach and install headers, Sergey branch + + cd $PREFIX/src + git clone https://git.savannah.gnu.org/git/hurd/gnumach.git + cd gnumach + git remote add bugaevc https://github.com/bugaevc/gnumach.git + git fetch bugaevc + git switch -c wip-aarch64 bugaevc/wip-aarch64 + autoreconf -i && mkdir build && cd build && CC=gcc ../configure + --prefix=$PREFIX --host=aarch64-gnu && make install-data + +Set PATH variable: + + PATH=$PATH:$PREFIX/bin + +Install mig + + $ cd $PREFIX/src + $ git clone https://git.savannah.gnu.org/git/hurd/mig.git + $ cd mig + $ autoreconf -i + $ mkdir build + $ cd build + $ ../configure --target=aarch64-gnu --prefix=$PREFIX + $ make + $ make install + +Install hurd headers + + $ cd $PREFIX/src + $ git clone https://git.savannah.gnu.org/git/hurd/hurd.git + $ cd hurd + $ git remote add bugaevc https://github.com/bugaevc/hurd.git + $ git fetch bugaevc + $ git switch -c aarch64 bugaevc/aarch64 + autoreconf -i && mkdir build && cd build + $ CC=gcc ../configure --host=aarch64-gnu --without-parted \ + --without-libcrypt--without-libbz2 --without-libz \ + --without-rump + $ make no_deps=t prefix=$PREFIX install-headers + +Install glibc headers + + $ cd $PREFIX/src + $ git clone https://sourceware.org/git/glibc.git + $ cd glibc/ + $ git remote add bugaevc https://github.com/bugaevc/glibc.git + $ git fetch bugaevc + $ git switch -c aarch64-gnu bugaevc/aarch64-gnu + $ mkdir build + $ cd build + $ ../configure --disable-mathvec --host=aarch64-gnu --prefix= \ + $ make install-headers DESTDIR=$PREFIX + $ touch $PREFIX/include/gnu/stubs.h $PREFIX/lib/libc.so + $ make csu/subdir_lib install csu/crt1.o csu/crti.o csu/crtn.o $PREFIX/lib + +Make libgcc + + $ cd $PREFIX/src + $ cd gcc/build + $ make configure-target-libgcc + $ make all-target-libgcc + $ make install-target-libgcc + +Make gnumach for real now + + $ cd $PREFIX/src + $ cd gnumach + $ rm -r build + $ mkdir build + $ cd build + $ ../configure --prefix=$PREFIX --host=aarch64-gnu + $ make + +Check booting notes in `aarch64/BOOTING`. + +Test gnumach: + + $ cd $PREFIX/src + $ cd gnumach/build + $ make tests/module-{hello,gsync,mach_host,machmsg,mach_port,syscalls,task,threads,vm,thread-state} + $ qemu-system-aarch64 -m 1G -machine virt -cpu max -kernel gnumach \ + -nographic -append "foo=bar" -device + guest-loader,kernel=tests/module-hello,bootargs="module-hello"'${host-port} + ${device-port} $(task-create) $(task-resume)',addr=0x4000 + for test in gsync hello mach_host machmsg mach_port syscalls task + threads vm; do qemu-system-aarch64 -m 1G -machine virt -cpu max + -kernel gnumach -nographic -append "foo=bar" -device + guest-loader,kernel=tests/module-$test,bootargs="module-$test + "'${host-port} ${device-port} $(task-create) + $(task-resume)',addr=0x4000; done + +## Debug gnumach: + + $ qemu-system-aarch64 -m 1G -machine virt -cpu max -kernel gnumach \ + -nographic -append "foo=bar" -device + guest-loader,kernel=tests/module-hello,bootargs="module-hello + "'${host-port} ${device-port} $(task-create) $(task-resume)',addr=0x4000 + +in another terminal (tab), run gdb, enter tar ext :1234, then p +$pc. $pc is in 0x40000000. Enter si 6 times, $pc should jump to +0x0000000040200000. symbol-file gnumach.elf -o $pc. + +## Build Das U-Boot + +Make Arm Trusted Firmware: + + $ cd $PREFIX/src + $ git clone https://github.com/ARM-software/arm-trusted-firmware.git + $ cd arm-trusted-firmware + $ export CROSS_COMPILE=aarch64-gnu- + $ export ARCH=arm64 + $ make PLAT=sun50i_a64 bl31 + +Make crust: + + $ cd $PREFIX/src + $ git clone https://github.com/crust-firmware/crust + $ cd crust + $ export CROSS_COMPILE=or1k-elf- + $ make a64-olinuxino_defconfig + $ make scp + +Download Das U-boot: + + $ cd $PREFIX/src + $ git clone https://github.com/u-boot/u-boot.git + +Make Das U-boot: + + $ cd $PREFIX/src + $ cd u-boot + $ export CROSS_COMPILE=aarch64-gnu- + $ export BL31=$PREFIX/src/arm-trusted-firmware/build/sun50i_a64/release/bl31.bin + $ export ARCH=arm64 + $ export SCP=$PREFIX/src/crust/build/scp/scp.bin + $ make distclean + $ make a64-olinuxino_defconfig + $ make all + +Install u-boot: + + # dd if=u-boot-sunxi-with-spl.bin of=/dev/[CHANGE THIS] bs=1024 seek=8 + +## Boot gnumach with Das U-boot + +### Boot from serial + +After copying dt.dtb from u-boot, gnumach and tests/module-hello from +gnumach in a local directory. Boot the system in u-boot. Then from a +serial terminal run: + + + loady 0x42000000 + <== load gnumach size:0x00069c58 + + loady 0x60000000 + <== load module-hello size:0x000779e0 + + loady 0x42200000 + <== load dt.dtb size:0x0000a018 + + fdt addr 0x42200000 + fdt resize + fdt set /chosen bootargs "foo=bar" + fdt set /chosen \#address-cells <0x2> + fdt set /chosen \#size-cells <0x2> + fdt mknod /chosen module@0x60000000 + fdt set /chosen/module@0x60000000 compatible "multiboot,kernel" "multiboot,module" + fdt set /chosen/module@0x60000000 reg <0x0 0x60000000 0x0 0x000779e0> + fdt set /chosen/module@0x60000000 bootargs "module-hello + "'${host-port} ${device-port} $(task-create) $(task-resume)' + + fdt print /chosen + + booti 0x42000000 - 0x42200000 + +Where <== load commands are terminal-emulator dependent. + +### Boot from smartcard + +Copy dt.dtb, gnumach, gnumach-tracing and all modules on the sd +card. The tracing version of gnumach is obtained by cherrypicking the +relative commit in a gnumach tree and rebuilding gnumach: + + git cherry-pick ee177f52680116538192b2c0c5d9a08e174c007f + +Create the file boot.cmd on the smartcard: + + if itest $trace != 0; then load mmc 0:1 0x42000000 gnumach-tracing; else load mmc 0:1 0x42000000 gnumach; fi + load mmc 0:1 0x60000000 $module + load mmc 0:1 0x42200000 dt.dtb + + fdt addr 0x42200000 + fdt resize + fdt set /chosen bootargs "foo=bar" + fdt set /chosen \#address-cells <0x2> + fdt set /chosen \#size-cells <0x2> + fdt mknod /chosen module@0x60000000 + fdt set /chosen/module@0x60000000 compatible "multiboot,kernel" "multiboot,module" + fdt set /chosen/module@0x60000000 reg <0x0 0x60000000 0x0 0x04000000> + fdt set /chosen/module@0x60000000 bootargs "$module "'${host-port} + ${device-port} $(task-create) $(task-resume)' + + booti 0x42000000 - 0x42200000 + +Compile it with: + + mkimage -C none -A arm -T script -d boot.cmd boot.scr + +Boot into Das u-boot, interrupt automatic booting to set the variables +before: + + trace=0 + module=module-hello + boot + +modify trace to a different number to start the version of gnumach +compiled with tracing. # IRC, freenode, #hurd, 2012-11-15 diff --git a/open_issues/audio.mdwn b/open_issues/audio.mdwn new file mode 100644 index 00000000..d31f9656 --- /dev/null +++ b/open_issues/audio.mdwn @@ -0,0 +1,115 @@ +[[!meta copyright="Copyright © 2024 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag stable_URL]] + +[[!tag open_issue_hurd]] + +The Hurd currently does not have an audio framework or [[audio device +drivers|hurd/rump/rumpsound]]. One option would be to port `ALSA` to +the Hurd, but `ALSA` is suboptimal for a [[number of +reasons|https://github.com/linuxaudio/Linux-Audio-Workgroup/wiki/Towards-Fixing-ALSA]]: + +* poor device enumeration. If you unplug a sound card and plug it in + again, you could get a different card number. The same problem can + happen when you reboot. +* With `ALSA` you cannot use two sound cards as one virtual sound card + with synced time clocks. +* `ALSA` makes it difficult to use a low latency application like + Ardour alongside a higher latency application, like a desktop + environment's sound notifications. +* `ALSA` encourages applications to control hardware directly using a + push model, meaning that the application tries to decide when to + play sound. The pull model is much better, in that the audio + subsystem tells hardware when to play sound based on requests from + applications. +* `ALSA` has backwards support for `OSS`, which is an even worse API, + which partially makes `ALSA` such a poor design. + +Pulseaudio and pipewire, which are built to support `ALSA`, have +similar problems. Our Hurd developer/audio-expert Damien Zammit +believes that the Hurd sound subsystem should use the [[`JACK` +daemon/API|https://logs.guix.gnu.org/hurd/2023-10-06.log]]. He is an +audiophile, who wrote and maintains some [[`JACK` +plugins|https://github.com/zamaudio/zam-plugins/releases]], and he +also wrote [[native `JACK` support for Qubes +OS|https://github.com/zamaudio/qubes-jack]]. Damien spoke to Paul +Davis, who is the reason there is professional audio on Linux, and the +author of Ardour DAW and JACK. Paul suggested we write a new JACK +backend directly with low level driver code, and hack on the driver +code until it is working. Then we could see how to define an API to +move the driver level code into a hurd translator. One potential +issue is that JACK uses lots of POSIX `shmem` that the hurd lacks, +though a patch exists in Debian or in the tg tree. + +We would have to create a hurd-jack backend for the `jackd` daemon. If +clients do not support the `JACK` API, then they can connect to the +Hurd's running pipewire daemon that would forward all sound requests +to the jackd daemon. We would have a hurd translator sitting on +`/dev/snd` and use RPCs to create a hurdish API that implements all of +the things that `jackd` needs. We would not use `ALSA`'s API +anywhere, but `ALSA`-only applications should be able to play sounds +through pipewire. + +A sound card driver would expose a set of RPCs that jackd would need. +We could implement that part ourselves, which would be the easy part. +The difficult part is making the driver, so that `jackd` opens +`/dev/snd` and calls the RPCs to control it. Audio will not flow +through `/dev/snd`. Rather, a callback is registered with a pointer +to some buffer. We would *NOT* move the audio data via Mach messages, +because that would be too slow. Instead jackd would `mmap` the device +buffer directly or use shared memory to a small ring-buffer. The +audio driver would `mmap` the device buffer directly, and we would +share that with the caller. + +The audio driver would start and stop the sound card. We just need to +fill its buffer somehow. The audio driver would know when to do this. +It would schedule a writer and a reader probably based on timers. The +arrival of the audio IRQ tells you when the hw pointer is at the end +of the cycle, but you don't just copy the buffer when this happens +(like alsa does). You use the timestamps of arrival of the audio IRQs +to calibrate a delay locked loop. At any time you want in between the +audio irq, you can ask the question: where is the buffer pointer now? + +We would use some kind of timer interrupt to schedule the transfer +when you want, depending on the requested latency. This all happens +in the audio driver, but it is controlled by `jackd`, and the audio +driver will call the callback provided by the user application. + +The audio driver basically writes and reads zeroes until you fill it +with some audio data, and if the callback blocks for too long and +cannot deliver the audio in time for the next cycle, the audio drops +the frame and you get silence, unless the audio driver is in the same +process as `jackd`. Somehow `jackd` gets access to a mmap-ed buffer. +`jackd` handles all the routing between different audio applications +by creating a process graph. + +[[rumpsound|hurd/rump/rumpsound]] has an audio driver, but it's based +on SunOS and [[does not have many +cards|https://logs.guix.gnu.org/hurd/2023-10-07.log]]. If rumpsound, +does not give the Hurd many device drivers for sound cards, then we +may be able to use `ALSA`'s well organized low-level functions that +implement all the features of sound cards. We will need to implement +a pcm streaming driver that calls into ALSA low-level functions (it +could be GPLv2). It's difficult to break out, so maybe we will just +start with pci. Damien got a sound card in qemu with `-soundhw ac97`, +and `linux/sound/pci/intel8x0.c` should support that. The tricky bit +will be sending audio data from the driver to another proccess without +`mach_msg ()`. We could possibly use a ring buffer. + +## irc logs + +[[https://logs.guix.gnu.org/hurd/2023-10-06.log|https://logs.guix.gnu.org/hurd/2023-10-06.log]] + +[[https://logs.guix.gnu.org/hurd/2023-10-07.log|https://logs.guix.gnu.org/hurd/2023-10-07.log]] + +[[https://logs.guix.gnu.org/hurd/2023-10-08.log|https://logs.guix.gnu.org/hurd/2023-10-08.log]] + + diff --git a/open_issues/bcachefs.mdwn b/open_issues/bcachefs.mdwn new file mode 100644 index 00000000..330a41e1 --- /dev/null +++ b/open_issues/bcachefs.mdwn @@ -0,0 +1,325 @@ +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +The Hurd's primary filesystem is ext2, which works but lacks modern +features. Ext2 does not have a journal, so Hurd users occasionally +have to deal with filesystem corruption. `fsck` can fix most of the +issues (with loss of random data), but without a proper journal the +Hurd currently is not a good a OS for long-term data storage. + +Bcachefs is a modern COW (copy-on-write) open source filesystem for +Linux, which intends to replace Btrfs and ZFS while having the +performance of ext4 or XFS. It is almost 100,000 lines of code. +Btrfs is 150,000 lines of code. Bcachefs is structured as a +filesystem built on top of a database. There is a clean small +database transaction layer. That core database library is maybe +25,000 lines of code. + +Some Hurd developers recently [[talked with +Bcachefs|https://youtube.com/watch?v=bcWsrYvc5Fg]] author Kent +Overstreat about porting bcachefs to the Hurd. There are currently no +concrete plans to do so due to lack of developer man power. + +90% of the Bcachefs filesystem code builds and runs in userspace. It +uses a shim layer that makes maps kernel locking primatives to +pthreads, the kernel io API is mapped to AIO, etc. Bcachefs does +intend to eventually rewrite most or all of its current codebase into +rust. + +Kent is ok with us merging a shim layer for libstore that maps to the +Unix filesystem API. That would be a header file that goes into the +bcachefs code. + +There is a somewhat working FUSE port of bcachefs, but Kent is not +certain that is a good way to run bcachefs in userspace. Kent wants +to use the FUSE port to help in debbugging. Suppose bcachefs starts +acting up, then you could switch to running it in userspace and attach +GDB to the running process. This is currently not possible. + +We could port bcachefs to the Hurd's native filesystem API: libdiskfs. + +One interesting aspect of the conversation was Kent's goal of re-using +kernel code in userspace. The Linux kernel hashtable code is high +performance, resizeable, lockless, and builds and runs in userspace. +As long as you have liburcu, then you can use the kernel hashtable in +userspace on the Hurd. This might be useful to use on the Hurd. + +Bcachefs is liscensed as GPLv2, and many of Kent's previous employers +own the patents, including Google. Kent is ok with potentially making +the license GPLv2+, as long as there was not a promise to keep +bcachefs GPLv2 only. + +# IRC logs + +https://logs.guix.gnu.org/hurd/2023-09-26.log + + <solid_black> maybe I'm wrong though, do you know much about fuse? or file systems? + <damo22> no i dont know much about filesystems + <damo22> what is bcachefs? + <solid_black> see? :D + <azert> I agree that someone intimate in the Mach pager api, libdiskfs and fuse would be great at that meeting + <solid_black> I do kind of understand Mach VM / paging, I must say + <solid_black> from the looks of it, I even understand it best among those who have looked at it recently + <solid_black> and I mostly understand libdiskfs + <damo22> so go to the meeting + <damo22> what is fuse? do we even need it for hurd? + <damo22> file systems in userspace + <solid_black> FUSE is "filesystem in user space", it's both the name for the concept, and the name of Linux's specific mechanism, of offloading fs to userland + <damo22> yeah, i think it may be unneeded for filesystem on hurd + <solid_black> it's basically a giant hack that pretends to be a fileystem implementation to the rest of the kernel, and then sends requests and receives responses from a userland program that _actually_ implements the fs + <solid_black> on the Hurd, *of course* filesystems are implemented in userland, that's the only and tnhe natural way everything works + <solid_black> but that's where the similarities end + <solid_black> you cannot just take a linux fuse fs, using libfuse, and run it on the Hurd + <solid_black> there has been a project make a library that would have the same API as libfuse, but act as a Hurd translator, specifically to facilitate porting linux filesystems + <damo22> i imagine fuse has an api + <solid_black> last I heard, it was never completed, but who knows + <solid_black> it has a kerne <->userland protocol and a userspace library (libfuse) for implementing that protocol, yes + <damo22> solid_black: you seem to know more about fuse than you admitted + <solid_black> https://www.gnu.org/software/hurd/hurd/libfuse.html + <solid_black> I know the basics, around as much as I have just told you + <azert> I think that gnucode idea was that this would be the easiest to port bcachefs to the Hurd, but I doubt it would be the best + <solid_black> I have also hacked on a C++ fuse fs (darling-dmg), though I don't think I interacted with the fuse parts of it much + <azert> Or even the easier + <solid_black> yeah, I don't think it'd be the best or the easiest one either + <damo22> if someone implemented libfuse api and made it as a hurd translator, surely it would work natively? + <damo22> <braunr> zacts: the main problem seems to be the interactions between the fuse file system and virtual memory (including caching) + <braunr> something the hurd doesn't excel at + <braunr> it *may* be possible to find existing userspace implementations that don't use the system cache (e.g. implement their own) + <azert> Yes, that’s a possibility that needs to be kept open for discussion + <nikolar> Sounds interesting + <solid_black> youpi: ping + <youpi> pong + <solid_black> hello! + <solid_black> any thoughts on the above discussion? are you going to participate in the call that's being set up? + <youpi> I don't have time for it + <youpi> (AFAIK the fuse hurd implementation does work to some extent) + <solid_black> I should at least try out Hurd's fuse before the call, good idea + <solid_black> maybe read up on the Linux's fuse + <solid_black> thoughts on using fuse vs libdiskfs for bcachefs? + <youpi> using fuse would probably be less work + <youpi> and it'd probably mean fixing things in libfuse, which can benefit many other FS anyway + <solid_black> is it true that the "low level" API of libfuse is unimplemented and unimplementable? + <youpi> I don't know what that "low level" API is + <solid_black> this IIUC https://github.com/libfuse/libfuse/blob/master/include/fuse_lowlevel.h + <solid_black> > libfuse offers two APIs: a "high-level", synchronous API, and a "low-level" asynchronous API. In both cases, incoming requests from the kernel are passed to the main program using callbacks. When using the high-level API, the callbacks may work with file names and paths instead of inodes, and processing of a request finishes when the callback function returns. When using the low-level API, the callbacks must work with inodes and responses must be se + <solid_black> nt explicitly using a separate set of API functions. + <youpi> where did you read that it'd be unimplementable ? + <solid_black> https://git.savannah.gnu.org/cgit/hurd/incubator.git/tree/README?h=libfuse/master + <solid_black> > This is simply because it is to specific to the Linux kernel and (besides that) it is not farly used now. + <youpi> In case the latter should change in the future, we might want to re-think about that issue though. + <solid_black> so, sounds like it's perhaps implementable in theory, but that'd require additional work and design + <youpi> see the sentence below... + <solid_black> the low-level API is what bcachefs uses + <youpi> well, additional work and design, of course + <solid_black> seems to, at least, from a quick glance + <youpi> any async API needs some + <youpi> but I don't see why it would not be possible + <youpi> mig precisely supports asynchronous stubs + <solid_black> bcachefs-tools/cmd_fusermount.c is just 1274 lines, which inspires some hope + <solid_black> asynchrony is not the problem, I imagine (but I haven't looked), but being too tied to Linux might be + <youpi> it's not really tied, as in it doesn't seem to use linux-specific functions + <youpi> but it uses linux-like notions, which indeed need to be translated to the hurdish notions + <youpi> but that's not something really tough + <youpi> just needs to be worked on + +https://logs.guix.gnu.org/hurd/2023-09-27.log#103329 + + <solid_black> libfuse as shipped as Debian doesn't seem very + functional, I can't even build a simple program against it: + 'i386-gnu/libfuse.so: undefined reference to `assert'' + + <solid_black> (assert is of course a macro in glibc) + <solid_black> and it segfaults in fuse_main_real + <solid_black> lowleve fuse ops do seem to map to netfs concept nicely, as far as I can see so far + <solid_black> and (again, so far) I don't see any asynchrony in how bcachefs uses fuse, i.e. they always fuse_reply() inside the method implementation + + <solid_black> but if we had to implement low-level fuse API, this would be an issue + <solid_black> because netfs is syncronous + <solid_black> this is again a place where I don't think netfs is actually that useful + <solid_black> libfuse should be its own standalone tranlator library, a peer to lib{disk,net,triv}fs + <solid_black> yell at me if you disagree + <youpi> or perhaps make it use libdiskfs ? + <youpi> there's significant code in libdiskfs that you'd probably not want to reimplement in libfuse + <solid_black> like what? + <youpi> starting a translator + <youpi> all the posix semantic bits + <solid_black> (this is another thing, I don't believe there is a significant difference that explains libdiskfs and libnetfs being two separate libraries. but it's too late to merge them, and I'm not an fs dev) + + <solid_black> starting a translator is abstracted into libfshelp specifically so it can be easily reused? + <solid_black> is libdiskfs synchronous? + <youpi> I'm just saying things out of my memory + <solid_black> scratch that, diskfs does not work like that at all + <youpi> piece of it is in fshelp yes + <solid_black> it works on pagers, always + <youpi> but significant pieces are in libdiskfs too + <youpi> and you are saying you are not an FS person :) + <youpi> you do know libdiskfs etc. well beyond the average + <youpi> perhaps not the ext2 FS structure, but that's not really important here + <youpi> see e.g. the short-circuits in file-get-trans.c + <solid_black> I may understand how the Hurd's translator libraries work, somewhat better than the avergae person :) + <youpi> and the code around fshelp_fetch_root + <solid_black> but I don't know about how filesystems are actually organized, on-disk (beyond the basics that there any inodes and superblocks and journaled writes and btrees etc) + <youpi> you don't really need to know more about that + <solid_black> nor do I know the million little things about how filesystem code should be written to be robust and performant + <solid_black> yeah so as I was saying, libdiskfs expects files to be mappable (diskfs_get_filemap_pager_struct), and then all I/O is implemented on top of that + <solid_black> e.g. to read, libdiskfs queries that pager from the impl, maps it into memory, and copies data from there to the reply message + <solid_black> I must have mentioned that already, I'd like to rewrite that code path some day to do less copying + <solid_black> I imagine this might speed up I/O heavy workloads + <youpi> ? it doesn't copy into the reply + <youpi> it transfers map + <solid_black> it does, let me find the code + <youpi> in some corner cases yes + <youpi> but not normal case + <youpi> https://darnassus.sceen.net/~hurd-web/hurd/io_path/ + <solid_black> libdiskfs/rdwr-internal.c, it does pager_memcpy, which is a glorified memcpy + fault handling + <solid_black> don't trust that wiki page + <youpi> why not ? + <youpi> not, pager_memcpy is not just a memcpy + <youpi> it's using vm_copy whenever it can + <youpi> i.e. map transfer + <solid_black> well yes, but doesn't the regular memcpy also attempt to do that? + <youpi> it happens to do so indeed + <youpi> but that' doesn't matter: I do mean it's trying *not* copying + <youpi> by going through the mm + <youpi> note: if a wiki page is bogus, propose a fix + <solid_black> I think there was another copy on the path somewhere (in the server, there's yet another in the client of course), but I can't quite remember where + <solid_black> and I wouldn't rely on that vm_copy optimization + <solid_black> it's may be useful when it working, but we have to design for there to not be a need to make a copy in the first place + <solid_black> ah well, pager_read_page does the other copy + <youpi> when things are not aligned etC. you'll have to do a copy anyway + <solid_black> but then again, this is all my idle observations, I'm not an fs person, I haven't done any profiling, and perhaps indeed all these copies are optimized away with vm_copy + <youpi> where in pager_read_page do you see a copy? + <youpi> it should be doing a store_read + <youpi> passing the pointer to the driver + <solid_black> ext2fs/pager.c:file_pager_read_page (at line 220 here, but I haven't pulled in a while) + <solid_black> it does do a store_read, and that returns a buffer, and then it may have to copy that into the buffer it's trying to return + <solid_black> though in the common case hopefully it'll read everything in a single read op + <youpi> it's in the new_buf != *buf + offs case + <youpi> which is not supposed to be the usual case + <solid_black> but now imagine how much overhead this all is + <youpi> what? the ifs? + <solid_black> we're inside io_read, we already have a buffer where we should put the data into + <youpi> I have to go give a course, gotta go + <solid_black> we could just device_read() into there + <youpi> you also want to use a cache + <youpi> otherwise it'll be the disk that'll kill yiour performance + <youpi> so at some point you do have to copy from the cache to the application + <youpi> that's unavoidable + <youpi> or if it's large, you can vm_copy + copy-on-write + <youpi> but basically, the presence of the cache means you can have to do copies + <youpi> and that's far less costly than re-reading from the disk + <solid_black> why can't you return the cache page directly from io_read RPC? + <youpi> that's vm_copy, yes + <youpi> but then if the app modifies the piece, you have to copy-on-write + <youpi> anywauy, really gottago + <solid_black> that part is handled by Mach + <solid_black> right, so once you're back: my conclusion from looking at libfuse is that it should be rewritten, and should not be using netfs (nor diskfs), but be its own independent translator framework + <solid_black> and it just sounds like I'm going to be the one who is going to do it + <solid_black> and we could indeed use bcachefs as a testbed for the low level api, and darling-dmg for the high level api + <solid_black> I installed avfs from Debian (one of the few packages that depend on libfuse), and sure enough: avfs: symbol lookup error: /lib/i386-gnu/libfuse.so.1: undefined symbol: assert_perror + <solid_black> upstream fuse is built with Meson 🤩️ + <solid_black> I'm wondering whether this would be better done as a port in the upstream libfuse, or as a Hurd-specific libfuse lookalike that borrows some code from the upstream one (as now) + <damo22> solid_black: what is your argument to rewrite a translator framework for fuse? + <damo22> i dont understand + <solid_black> hi + <damo22> hi + <solid_black> basically, 1. while the concepts of libfuse *lowlevel* api seem to match that of hurd / netfs, they seem sufficiently different to not be easily implementable on top of netfs + <solid_black> particularly, the async-ness of it, while netfs expects you to do everything synchronously + <damo22> is that a bug in netfs? + <solid_black> this could be maybe made to work, by putting the netfs thread doing the request to sleep on a condition variable that would get signalled once the answer is provided via the fuse api... but I don't think that's going to be any nicer than designing for the asynchrony from the start + <solid_black> it's not a bug, it's just a design decision, most Hurd tranalators are structured that way + <damo22> maybe you can rewrite netfs to be asynchronous and replace it + <solid_black> i.e.: it's rare that translators use MIG_NO_REPLY + explicit reply, it's much more common to just block the thread + <solid_black> 2. the current state is not "somewhat working", it's "clearly broken" + <damo22> why not start by trying to implement rumpdisk async + <damo22> and see what parts are missing + <solid_black> wdym rumpdisk async? + <damo22> rumpdisk has a todo to make it asynchronous + <damo22> let me find the stub + <damo22> * FIXME: + <damo22> * Long term strategy: + <damo22> * + <damo22> * Call rump_sys_aio_read/write and return MIG_NO_REPLY from + <damo22> * device_read/write, and send the mig reply once the aio request has + <damo22> * completed. That way, only the aio request will be kept in rumpdisk + <damo22> * memory instead of a whole thread structure. + <solid_black> ah right, that reminds me: we still don't have proper mig support for returning errors asynchronously + <damo22> if the disk driver is not asynchronous, what is the point of making the filesystem asynchronous? + <solid_black> the way this works, being asynchronous or not is an implementatin detail of a server + <solid_black> it doesn't matter to others, the RPC format is the same + <solid_black> there's probably not much point in asynchrony for a real disk fs like bcachefs, which must be why they don't use it and reply immediately + <solid_black> but imagine you're implementing an over-the-network fs with fuse, then you'd want asynchrony + <damo22> what is your goal here? do you want to fix libfuse? + <solid_black> I don't know + <solid_black> I'm preparing for the call with Kent + <solid_black> but it looks like I'm going to have to rewrite libfuse, yes + <damo22> possibly the caching is important + <damo22> ie, where does it happen + <solid_black> maybe, yes + <solid_black> does fuse support mmap? + <damo22> idk + <damo22> good q for kent + <solid_black> one essential fs property is coherence between mmap and r/w + <solid_black> so it you change a byte in an mmaped file area, a read() of that byte after that should already return the new value + <solid_black> same for write() + read from memory + <solid_black> this is why libdiskfs insists on reading/writing files via the pager and not via callbacks + <solid_black> I wonder how fuse deals with this + <damo22> good point, no idea + <solid_black> does fuse really make the kernel handle O_CREAT / O_EXCL? I can't imagine how that would work without racing + <solid_black> guess it could be done by trying opening/creating in a loop, if creation itself is atomic, but this is not nice + <damo22> something is still slowing down smp + <damo22> it cant possibly be executing as fast as possible on all cores + <damo22> if more cores are available to run threads, it should boot faster not slower + <azert> Hi damo22, your reasoning would hold if the kernel wouldn’t be “wasting” most of its time running in kernel mode tasks + <azert> If replacing CPU_NUMBER by a better implementation gave you a two digits improvement, that kind of implies that the kernel is indeed taking most of the cpu + <damo22> yes i mean, something in the kernel is slowing down smp + <azert> What about vm_map and all thread tasks synchronization + <azert> ? + <damo22> i dont understand how the scheduler can halt the APs in machine_idle() and not end up wasting time + <damo22> how does anything ever run after HLT + <damo22> in that code path + <damo22> if the idle thread halts the processor the only way it can wake up is with an interrupt + <damo22> but then, does MARK_CPU_ACTIVE() ever run? + <damo22> hmm it does + <azert> I think that normally the cpu would be running scheduler code and get a thread by itself. + <damo22> thats not how it works + <damo22> most of the cpus are in idle_continue + <damo22> then on a clock interrupt or ast interrupt, they are woken to choose a thread i think + <damo22> s/choose/run + <azert> If they are in cpu_idle then that’s what happens, yea + <azert> But normally they wouldn’t be in cpu idle but running the schedule and just a thread on their own + <azert> Cpu_idle basically turns off the cpu + <azert> To save power + <damo22> every time i interrupt the kernel debugger, its in cpu-idle + <damo22> i dont know if it waits until it is in that state so maybe thats why + <azert> That means that there is nothing to schedule + <azert> Or yea that’s another explanation + <damo22> yes, exactly i think it is seemingly running out of threads to schedule + <azert> A bug in the debugger + <damo22> i need to print the number of threads in the queue + <youpi> adding a show subcommand for the scheduler state would probably be useful + <youpi> solid_black: btw, about copies, there's a todo in rumpdisk's rumpdisk_device_read : /* directly write at *data when it is aligned */ + <solid_black> youpi: indeed, that looks relevant, and wouldn't be hard to do + <solid_black> ideally, it should all be zero-copy (or: minimal number of copies), from the device buffer (DMA? idk how this works, can dma pages be then used as regular vm pages?) all the way to the data a unix process receives from read() or something like that + <solid_black> without "slow" memcpies, and ideally with little vm_copies too, though transferring ages in Mach messages is ok + <solid_black> s/ages/pages/ + <solid_black> read() requires ones copy purely because it writes into the provided buffer (and not returns a new one), and we don't have mach_msg_overwrite + <solid_black> though again one would hope vm_copy would help there + <solid_black> ...I do think that it'd be easier to port bcachefs over to netfs than to rewrite libfuse though + <solid_black> but then nothing is going to motivate me to work on libfuse + <azert> solid_black: I never work on things that don’t motivate me somehow + <azert> Btw, if you want zerocopy for IO, I think you need to do asynchronous io + <azert> At least that’s the only way for me to make sense of zerocopy + <solid_black> I don't think sync vs async has much to do with zero-copy-ness? w + + diff --git a/open_issues/clock_gettime.mdwn b/open_issues/clock_gettime.mdwn deleted file mode 100644 index 407a104c..00000000 --- a/open_issues/clock_gettime.mdwn +++ /dev/null @@ -1,348 +0,0 @@ -[[!meta copyright="Copyright © 2011, 2013, 2014 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!meta title="clock_gettime"]] - -[[!tag open_issue_glibc open_issue_gnumach]] - -Missing `clock_gettime(CLOCK_MONOTONIC)` (e.g. for iceweasel) - -It could be a mere matter of extending the -[[mapped-time_interface|microkernel/mach/gnumach/interface/device/time]]: -add it to -`mapped_time_value_t` in gnumach, handle it in `gnumach/kern/mach_clock.c`, and -make `clock_gettime` use it. - -BTW, also make `gettimeofday()` use it, since it's way more efficient and some -applications assume that it is. - -What about adding a nanosecond-precision clock, too? --[[tschwinge]] - - -# IRC, freenode, #hurd, 2011-08-26 - - < pinotree> youpi: thing is: apparently i found a simple way to have a - monotonic clock as mmap-able device inside gnumach - < pinotree> currently, in kern/mach_clock.c there's a variable 'time', - which gets increased on clock interrupt, and optionally modified by - host_set_time - < pinotree> () - < pinotree> if i add a new variable next to it, only increasing it on - interrupt but not modifying it at all otherwise, would that give me a - monotonic clock? - < pinotree> at least on sme basic tests i did, it seems it could work that - way - < youpi> yes, it should work - < braunr> sure - < youpi> and that's the way I was considering implementing it - - -# IRC, freenode, #hurd, 2011-09-06 - - <pinotree> yeah, i had a draft of improved idea for also handling - nanoseconds - <tschwinge> pinotree: Ah, nice, I thought about nanoseconds as well. - <tschwinge> pinotree, youpi: This memory page is all-zero by default, - right? - <tschwinge> Can't we then say that its last int is a version code, and if - it is 0 (as it is now), we only have the normal mapped time field, if it - is 1, we also have the monotonic cliock and ns precision on address 8 and - 16 (or whatever)? - <tschwinge> In case that isn't your plan anyway. - <youpi> it's all-zero, yes - <tschwinge> Or, we say if a field is != 0 it is valid. - <youpi> making the last int a version code limits the size to one page - <youpi> I was thinking a field != 0 being valid is simpler - <youpi> but it's probably a problem too - <youpi> in that glibc usually caches whether interfaces are supported - <tschwinge> Wrap-around? - <youpi> for some clocks, it may be valid that the value is 0 - <youpi> wrap-around is another issue too - <tschwinge> Well, then we can do the version-field thing, but put it right - after the current time field (address 8, I think)? - <youpi> yes - <youpi> it's a bit ugly, but it's hidden behind the structure - <tschwinge> It's not too bad, I think. - <youpi> yes - <tschwinge> And it will forever be a witness of the evolving of this - map_time interface. :-) - - -# IRC, freenode, #hurd, 2013-02-11 - -In context of [[select]]. - - <pinotree> braunr: would you send for review (and inclusion) your - time_data_t addition? - <pinotree> this way we could add nanosecs-based utime rpc (and then their - implementation in libc) - <braunr> pinotree: it's part of the hurd branch - <braunr> do you want it sent separately ? - <pinotree> yeah - <braunr> ok - <braunr> let me get it right first :) - <pinotree> sure :) - - -## IRC, freenode, #hurd, 2013-02-12 - - <braunr> pinotree: - http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout_pthread_v2&id=6ec50e62d9792c803d00cbff1cab2c0b3675690a - <pinotree> uh nice - <pinotree> will need two small inline functions to convert time_data_t <-> - timespec, but that's it - <braunr> hm right - <braunr> i could have thought about it - <braunr> but i'll leave it for another patch :p - <pinotree> oh sure, no hurry - - -## IRC, freenode, #hurd, 2013-02-19 - - <youpi> braunr: about time_data_t, I get it's needed that it be an array - <youpi> so it can be passed by reference, not by value? - <braunr> by address, yes - <braunr> that's the difference between array and struct - - -## IRC, freenode, #hurd, 2013-02-25 - - <youpi> braunr: why did you want to see time_data passed as pointer, not as - struct? - <braunr> to microoptimize - <braunr> the struct is 2 64-bit integers - <youpi> well, we already pass structs along in a few cases, - e.g. io_statbuf_t, rusage_t, etc. - <youpi> be it written t[0].sec or t->sec, it seems odd - <youpi> copying 2 64bit integers is not much compared to the potential for - bugs here - <braunr> bugs ? - <youpi> yes, as in trying to access t[1], passing a wrong pointer, etc. - <youpi> or the reader frowning on "why is this case different than the - others?" - <braunr> well, i'm already usually frowning when i see what mig does .. - <youpi> right - <youpi> on the plus side, it's only the client side, i.e. mostly glibc, - which sees the t[0] - <braunr> and the practice established by my patch is to convert to struct - timespec as soon as possible - <braunr> the direct use of this type is therefore limited - <youpi> could we define time_data_t as a struct time_data * instead of - struct time_data[1] ? - <youpi> (in the.h) - <youpi> that would make more sense to define a struct time_data, and pass a - pointer to it - <braunr> i'm not sure - <braunr> the mach server writing guide was very clear about array implying - a C array too - <braunr> and i remember having compilation problems before doing that - <braunr> but i don't remember their nature exactly - <youpi> I'm not sure to understand what you said about converting to struct - timespec - <youpi> what makes it not possible now? - <youpi> and what is the relation with being an array or a pointer? - <braunr> concerning struct timespec, what i mean is that the functions - called by the mig stub code directly convert time_data_t to a struct - timespec (which is the real type used throughout the hurd code) - <braunr> about the rest, i'm not sure, i'd have to try again - <braunr> mig just assumes it's an array - <youpi> and why not just using struct timespec? - <youpi> (for the mig type too) - <braunr> my brain can't correctly compute variable sized types in mig - definition files - <braunr> i wanted something that would remain correct for the 64-bit port - -[[64-bit_port]], [[mig_portable_rpc_declarations]]. - - <youpi> ah, you mean because tv_nsec is a long, which will not be the same - type? - <braunr> and tv_sec being a time_t (thus a long too) - <youpi> but we have the same issue e.g. for the rusage structure, don't we? - <braunr> yes - <youpi> so we'll have to fix things for that too anyway - <braunr> sure - <youpi> making a special case will not necessarily help - <braunr> but it doesn't mean new interfaces have to be buggy too - <youpi> well, using the proper type in the server itself is nicer - <youpi> instead of having to convert - <braunr> yes - <braunr> i'm not exactly sure where to declare struct timespec then - <braunr> should it be declared in hurd_types.h, and simply reused by the - libc headers ? - <youpi> ? AIUI, it's the converse, hurd_types.h uses the struct timespec - from libc headers, and defines timespec_t - <braunr> ok - <youpi> timespec_t being the internal type whose definition gets done right - for mig to do the right thing - <braunr> yes - <braunr> i see - <braunr> so, you'd like a struct of integer_t instead of an array of - signed64 - <youpi> for our current 32bit userland yes - <braunr> do you want to make the changes yourself or should i add a new - branch ? - <youpi> and we'll make that a 64bit struct when we have a64bit userland - - -# IRC, freenode, #hurd, 2013-04-06 - - <tschwinge> pinotree: You had once been working on adding nsec-procision - timestamps to GNU Mach's maptime interface (or what the name is). Is - that blocked on something or just waiting to be continued? - <pinotree> blocked on me needing to learn more the proper way to do - "atomic" update of the struct with time :) - - -# IRC, freenode, #hurd, 2013-09-04 - - <teythoon> do we have CLOCK_MONOTONIC ? - <braunr> teythoon: i think we do but it's actually a simple offset from - CLOCK_REALTIME .. :) - <teythoon> ah never mind, I do hate this posix time interface anyways - <braunr> really ? - <braunr> i think librt is decent - - -# Candidate for [[vDSO]] code? - - -# IRC, freenode, #hurd, 2014-02-23 - - <desrt> GLib (gthread-posix.c): Unexpected error from C library during - 'pthread_condattr_setclock': Invalid argument. Aborting. - <desrt> uh oh... - <desrt> time to go digging in glibc i guess... - <braunr> what are you trying to run ? - <desrt> glib - <braunr> with what ? - <desrt> just running glib's test suite under jhbuild - <desrt> i maintain glib and i made some changes recently -- i wanted to - make sure they didn't break the hurd - <desrt> and it seems they have ;/ - <braunr> well - <braunr> the hurd doesn't completely comply with posix 2008 - <desrt> long story short: we've keyed our timed waits on condition - variables to the monotonic clock for a long time now, but we never tested - that it actually worked - <desrt> so i just added an assert -- and indeed it fails on hurd - <braunr> our glibc lies about supporting timers - <braunr> good thinking - <braunr> we don't support the monotonic clock - <desrt> clock_gettime(CLOCK_MONOTONIC) seems to work - <braunr> and you should know that, even if clock selection and timers are - available (which posix 2008 requires), it's still optional - <braunr> no, glibc lies - <desrt> !! - <braunr> our "support" is a mere hack shifting CLOCK_REALTIME - <desrt> it should at least lie consistently :) - <braunr> we need to implement CLOCK_MONOTONIC properly - <desrt> ya... that would be very nice indeed - <braunr> not that hard either - <desrt> i agree! - <braunr> we just have to do it right - <desrt> fwiw, i plan to keep this assert in glib - <braunr> yes, it's good - <desrt> is there anywhere i can file a bug to give you guys some advance - warning? - <braunr> i don't think it's needed - <braunr> we know the problem - <desrt> k -- consider yourself warned, then :) - <braunr> and it's been a bigger concern recently - <desrt> awesome. glad i don't have to do anything :) - <braunr> if it's not already done, i suggest you check for the - CLOCK_MONOTONIC option - <desrt> fwiw, i'm trying to get a regular debian/gnu/hurd build of - glib/gtk/etc setup - <braunr> regular ? - <desrt> ya... out of git master on a daily basis - <braunr> from sources ? - <braunr> oh nice - <desrt> we recently set this up for freebsd as well - <braunr> few maintainers take the pain :) - <desrt> our non-linux 'problem discovery' is a bit crap before now :/ - <braunr> i guess that's pretty normal - <braunr> i don't consider it the responsibility of the maintainers to test - every possible platform - <desrt> glib is a bit unique -- portability is our business - <braunr> taking our patches into consideration is what we ask most - <braunr> right - <desrt> and the "please take the patches" thing is something we want to - stop doing - <braunr> why ? - <desrt> mostly because we often look at a patch that someone sent a few - years ago and say "do we even still need this?" - <desrt> and have no way to know - <braunr> uh - <desrt> you would not believe how many patches like this we've - accumulated... - <braunr> but if we send it now ? :) - <desrt> braunr: new policy is roughly this: - https://wiki.gnome.org/Projects/GLib/SupportedPlatforms - <desrt> ie: fixes for issues that are general portability improvements and - POSIX compliance are welcome... - <desrt> patches that introduce platform-specific #ifdef sections are - rejected unless we have a regular builder to test that code - <braunr> i see - <braunr> again, regarding portability, don't consider CLOCK_MONOTONIC to be - readily available, check for it - <braunr> an #error would be enough but it has to be checked - <desrt> it basically comes down to: we don't want to have code in our - version control that we have no possible way of testing - <braunr> yes - <desrt> braunr: we do check for it - <braunr> ok - <desrt> we assert() if clock_gettime(CLOCK_MONOTONIC) fails - <braunr> no i mean - <desrt> as POSIX said it should if CLOCK_MONOTONIC is not supported - <desrt> if you lie to us.... well, not much we can do - <braunr> POSIX_MONOTONIC_CLOCK - <braunr> _POSIX_MONOTONIC_CLOCK - <desrt> this is actually defined to 0 on most platforms... - <desrt> which does not mean that it's unsupported -- it means that the - runtime must be ready to deal with it not actually existing at runtime - <braunr> really ? - <desrt> yes - <desrt> we used to rely on this and got a bug that we were doing it wrong - :) - <desrt> and indeed, even on linux, both with glibc and uclibc: - <desrt> /usr/include/bits/posix_opt.h:#define _POSIX_MONOTONIC_CLOCK - 0 - <desrt> /usr/include/uClibc/bits/posix_opt.h:#define _POSIX_MONOTONIC_CLOCK - 0 - <braunr> ok it's described in 2.1.6 Options - <braunr> so your check is appropriate - <desrt> so does clock_gettime(MONOTONIC) on debian/hurd get me realtime? - <braunr> either that, or a value shifted from it - <desrt> if so, i'll just hack out the condattr_setclock() check and proceed - trying to build past glib... - * desrt checks - <desrt> as it is, even the build of glib fails since we use some tools - linked against ourselves during the build process... - <desrt> 1393124084790000 1393124084790000 - <desrt> those look the same.... - <braunr> heh - <desrt> i also notice that your clocks are not very high precision :) - <braunr> that's right - <desrt> HZ = 100, i guess - <braunr> yes - <desrt> fair enough - <desrt> our mainloop doesn't support better-than-millisecond accuracy yet - anyway :) - <desrt> (although it will soon...) - <braunr> nice - - -## IRC, freenode, #hurd, 2014-03-05 - - <desrt> braunr: bit of a warning: i released the glib that depends on - working pthread_condattr_setclock(..._MONOTONIC) and pochu said that it - will be landing in debian within the next days - <braunr> desrt: ok diff --git a/open_issues/copyright_assignment.mdwn b/open_issues/copyright_assignment.mdwn index dd5fa3d4..bc3e66f7 100644 --- a/open_issues/copyright_assignment.mdwn +++ b/open_issues/copyright_assignment.mdwn @@ -15,6 +15,5 @@ Current issues with copyright assignments: * Kalle Olavi Niemitalo ([contribution](https://savannah.gnu.org/bugs/index.php?48930) [contribution](https://savannah.gnu.org/bugs/index.php?49056)) [does not want to assign copyright](https://lists.gnu.org/archive/html/bug-hurd/2016-10/msg00069.html) * Olaf Buddenhagen, faced with process issues, gave up around 2016 july 23rd. * Brent Baccala (cases #1160318 #1497319): does not want to assign all past+future changes, would rather provide code under a BSD licence, or assign copyright separately for each contribution (request-assign.changes). -* Sergey Bugaev (case #1722866): sent form on 2021 May 8th, got answer on May 24th, immediately answered that there was an error in the actual family name. Eventually that was fixed, but the papers have the name in an order which is not proper for Russian law. Moved on with it. But hasn't had any answer during august and september. And then he got conscribed and so we won't have any news from him until around november 2022. * Luca Dariz (case #1803259): process ongoing, sent form on 2022 Feb 1st, got answer on Feb 11th, pending completion. * Various main contributors consider copyright assignment an unnecessary burden, the Hurd being considered as never-to-be-key-software. diff --git a/open_issues/emacs.mdwn b/open_issues/emacs.mdwn index 298ebc8c..15a7c761 100644 --- a/open_issues/emacs.mdwn +++ b/open_issues/emacs.mdwn @@ -12,27 +12,34 @@ License|/fdl]]."]]"""]] [[!tag open_issue_porting]] -GNU Emacs mostly does work, however there are a few issues. +GNU Emacs works pretty well. It works beautifully in X. --- -# 2023-08-31 +# 2024-01-14 -Emacs version 27.1 runs on Debian GNU/Hurd, but it does not run super -smoothly on the Hurd console. Emacs will run in the Hurd console, but -not as well. These three commands kill emacs in the Hurd console -pretty reliably: +Emacs version 29.1 runs on Debian GNU/Hurd. Joshua Branson uses it in +X without issue. He use org-mode, gnus, erc, calc, markdown-mode, etc. +In fact he, edits this wiki via Debian GNU/Hurd (running on a T43) +using Emacs. He is able to make changes to the wiki, render the wiki, +look at the resulting webpage with the +[[netsurf|https://www.netsurf-browser.org/]] web browser, commit the +changes, and send a patch to `bug-hurd@gnu.org`, all while using the +Hurd. :) - (use-package magit :ensure t) - M-x package-list-packages - M-x find-file ~/.authinfo.gpg +Also since the Hurd ported rust, you can now use +[[treesitter|https://www.masteringemacs.org/article/how-to-get-started-tree-sitter]] +with Emacs! Joshua got +[[html-ts-mode|https://github.com/mickeynp/html-ts-mode]] working. -Emacs does runs much better under X. Most things work well: org-mode, -gnus, erc, etc. +Emacs also has a packaged eglot, which is a client for language server +protocol, into core. Joshua was able to get eglot working for C mode +with clandg, but it was a bit slow. Joshua is running Emacs on a T43, +with 1.5 GB of RAM, so perhaps his hardware is just super slow. You can easily install many Emacs packages with apt: # apt install elpa-zenburn-theme - +Or you can use `use-package`. diff --git a/open_issues/git-core-2.mdwn b/open_issues/git-core-2.mdwn index 5afda5e8..e973ae10 100644 --- a/open_issues/git-core-2.mdwn +++ b/open_issues/git-core-2.mdwn @@ -76,7 +76,7 @@ Still seen. out that both clones do have some decompression errors. (Some files are missing chunks in /both/ cloned repositories). <braunr> where did you clone it from ? - <sea`> git.sv.gnu.org/hurd/hurd.git + <sea`> git.savannah.gnu.org/hurd/hurd.git <braunr> hum decompression errors ? <braunr> can you paste them please ? <sea`> Hmm, I can clone again and show you an example if I find one diff --git a/open_issues/images/overview.svg b/open_issues/images/overview.svg index bd47da91..0098d12c 100644 --- a/open_issues/images/overview.svg +++ b/open_issues/images/overview.svg @@ -2,23 +2,70 @@ <!-- Created with Inkscape (http://www.inkscape.org/) --> <svg - xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" - xmlns:dc="http://purl.org/dc/elements/1.1/" - xmlns:cc="http://creativecommons.org/ns#" - xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" - xmlns:svg="http://www.w3.org/2000/svg" - xmlns="http://www.w3.org/2000/svg" - xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" - xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" width="180.62869mm" height="150.9025mm" viewBox="0 0 180.62869 150.9025" version="1.1" id="svg8" - inkscape:version="0.92.3 (2405546, 2018-03-11)" - sodipodi:docname="overview.svg"> + inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)" + sodipodi:docname="overview.svg" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns="http://www.w3.org/2000/svg" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:dc="http://purl.org/dc/elements/1.1/"> <defs id="defs2"> + <rect + x="153.7989" + y="512.664" + width="133.75775" + height="30.069179" + id="rect15995" /> + <rect + x="70.616629" + y="204.61496" + width="133.62723" + height="25.853795" + id="rect15064" /> + <inkscape:path-effect + effect="spiro" + id="path-effect5506" + is_visible="true" + lpeversion="1" /> + <rect + x="71.043527" + y="502.39118" + width="118.31473" + height="31.646205" + id="rect5138" /> + <rect + x="433.82533" + y="435.59431" + width="188.13058" + height="23.777902" + id="rect2106" /> + <rect + x="277.88784" + y="434.10435" + width="153.88114" + height="30.839844" + id="rect652" /> + <inkscape:perspective + sodipodi:type="inkscape:persp3d" + inkscape:vp_x="0 : 75.45125 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_z="180.62869 : 75.45125 : 1" + inkscape:persp3d-origin="90.314345 : 50.300833 : 1" + id="perspective630" /> + <rect + x="53.75" + y="475.92355" + width="158.7221" + height="36.727121" + id="rect514" /> <inkscape:path-effect effect="bspline" id="path-effect5670" @@ -33,10 +80,14 @@ inkscape:isstock="true" style="overflow:visible" id="marker5632" - refX="0" - refY="0" - orient="auto" - inkscape:stockid="Arrow2Lend"> + refX="-4.8904819" + refY="-1.3198085" + orient="178.36419678" + inkscape:stockid="Arrow2Lend" + viewBox="0 0 12.705841 9.5264135" + markerWidth="19.705999" + markerHeight="14.774898" + preserveAspectRatio="xMidYMid"> <path transform="matrix(-1.1,0,0,-1.1,-1.1,0)" d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" @@ -122,14 +173,14 @@ inkscape:stockid="Arrow1Lend" orient="auto" refY="0" - refX="0" + refX="-5" id="marker5138" style="overflow:visible" inkscape:isstock="true"> <path id="path5136" d="M 0,0 5,-5 -12.5,0 5,5 Z" - style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" transform="matrix(-0.8,0,0,-0.8,-10,0)" inkscape:connector-curvature="0" /> </marker> @@ -219,19 +270,6 @@ transform="matrix(0.8,0,0,0.8,10,0)" inkscape:connector-curvature="0" /> </marker> - <linearGradient - inkscape:collect="always" - id="linearGradient4525" - osb:paint="gradient"> - <stop - style="stop-color:#000000;stop-opacity:1;" - offset="0" - id="stop4521" /> - <stop - style="stop-color:#000000;stop-opacity:0;" - offset="1" - id="stop4523" /> - </linearGradient> </defs> <sodipodi:namedview id="base" @@ -240,9 +278,9 @@ borderopacity="1.0" inkscape:pageopacity="0.0" inkscape:pageshadow="2" - inkscape:zoom="0.98994949" - inkscape:cx="319.89349" - inkscape:cy="299.72889" + inkscape:zoom="0.655" + inkscape:cx="324.42748" + inkscape:cy="277.09924" inkscape:document-units="mm" inkscape:current-layer="layer1" showgrid="false" @@ -251,11 +289,15 @@ fit-margin-right="0" fit-margin-bottom="0" inkscape:snap-global="false" - inkscape:window-width="1278" - inkscape:window-height="758" + inkscape:window-width="1276" + inkscape:window-height="746" inkscape:window-x="0" - inkscape:window-y="20" - inkscape:window-maximized="1" /> + inkscape:window-y="0" + inkscape:window-maximized="0" + inkscape:showpageshadow="2" + inkscape:pagecheckerboard="0" + inkscape:deskcolor="#d1d1d1" + showguides="true" /> <metadata id="metadata5"> <rdf:RDF> @@ -264,7 +306,7 @@ <dc:format>image/svg+xml</dc:format> <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> - <dc:title></dc:title> + <dc:title /> </cc:Work> </rdf:RDF> </metadata> @@ -274,21 +316,22 @@ id="layer1" transform="translate(-24.085951,-107.64845)"> <path - style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.96499997;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" - d="m 33.56429,116.78091 h 163.66374 v 134.9375 H 33.56429 Z" + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:1.00919;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + d="M 30.27983,112.3293 H 199.85767 V 254.76074 H 30.27983 Z" id="rect3713" inkscape:connector-curvature="0" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="79.31002" - y="128.01953" - id="text4539"><tspan + style="font-style:normal;font-weight:normal;font-size:10.2234px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.255585" + x="81.945656" + y="118.79103" + id="text4539" + transform="scale(0.96792078,1.0331424)"><tspan sodipodi:role="line" id="tspan4537" - x="79.31002" - y="128.01953" - style="stroke-width:0.26458332">The GNU/Hurd</tspan></text> + x="81.945656" + y="118.79103" + style="stroke-width:0.255585">The GNU/Hurd</tspan></text> <flowRoot xml:space="preserve" id="flowRoot4541" @@ -299,42 +342,44 @@ height="237.14285" x="80" y="178.91158" /></flowRegion><flowPara - id="flowPara4547"></flowPara></flowRoot> <rect + id="flowPara4547" /></flowRoot> + <rect style="fill:#86ee73;fill-opacity:1;stroke:#000000;stroke-width:0.5;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect4553" width="153.78763" height="42.85815" - x="38.758991" - y="178.51956" + x="38.248131" + y="172.38937" inkscape:transform-center-x="4.3782942" inkscape:transform-center-y="2.4590809" /> <rect - style="fill:#c7c5f8;fill-opacity:1;stroke:#000000;stroke-width:0.47089785;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + style="fill:#c7c5f8;fill-opacity:1;stroke:#000000;stroke-width:0.263752;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect4555" - width="65.128036" - height="17.881008" - x="81.836281" - y="229.65338" /> + width="34.778923" + height="10.50468" + x="94.432724" + y="225.66533" /> <rect style="fill:#e7ee73;fill-opacity:1;stroke:#000000;stroke-width:0.5;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect4557" width="154.77304" height="15.707925" x="38.28307" - y="154.56268" /> + y="150.85857" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="91.586601" - y="242.5256" - id="text4569"><tspan + style="font-style:normal;font-weight:normal;font-size:5.98974px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.149744" + x="103.41888" + y="219.21042" + id="text4569" + transform="scale(0.93964447,1.0642323)"><tspan sodipodi:role="line" - x="91.586601" - y="242.5256" - style="stroke-width:0.26458332" - id="tspan4571">Gnumach</tspan></text> + x="103.41888" + y="219.21042" + style="stroke-width:0.149744" + id="tspan455">GNU Mach</tspan></text> <rect - y="132.08675" + y="124.67848" x="38.373459" height="16.92679" width="154.59225" @@ -345,11 +390,11 @@ id="rect4585" width="51.048477" height="12.828938" - x="51.256123" - y="188.83125" /> + x="49.736835" + y="179.42331" /> <rect - y="205.80225" - x="126.92694" + y="196.62552" + x="131.30467" height="12.828938" width="51.048477" id="rect4587" @@ -359,11 +404,11 @@ id="rect4589" width="51.048477" height="12.828938" - x="51.507275" - y="205.48126" /> + x="49.985035" + y="196.81154" /> <rect - y="188.83125" - x="126.92694" + y="179.65126" + x="130.94589" height="12.828938" width="51.048477" id="rect4591" @@ -371,36 +416,36 @@ <flowRoot xml:space="preserve" id="flowRoot4593" - style="font-style:normal;font-weight:normal;font-size:26.66666794px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" - transform="matrix(0.26458333,0,0,0.26458333,41.332896,100.22933)"><flowRegion - id="flowRegion4595" - style="font-size:26.66666794px"><rect + style="font-style:normal;font-weight:normal;font-size:21.3333px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + transform="matrix(0.26458333,0,0,0.26458333,45.758165,93.384506)"><flowRegion + id="flowRegion4595"><rect id="rect4597" width="157.7106" height="30.868303" x="196.97975" - y="299.61926" - style="font-size:26.66666794px" /></flowRegion><flowPara - id="flowPara4599">Hurd Servers</flowPara></flowRoot> <text + y="299.61926" /></flowRegion><flowPara + id="flowPara4599" + style="font-size:21.3333px">Hurd Servers Servers</flowPara></flowRoot> + <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:7.05555534px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" x="95.447365" - y="139.10298" + y="131.69475" id="text4603"><tspan sodipodi:role="line" id="tspan4601" x="95.447365" - y="139.10298" - style="font-size:7.05555534px;stroke-width:0.26458332">Applications</tspan></text> + y="131.69475" + style="font-size:7.05556px;stroke-width:0.264583">Applications</tspan></text> <rect style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.5;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect4605" width="45.435822" height="8.9989662" x="43.772575" - y="136.02699" /> + y="128.61876" /> <rect - y="136.02699" + y="128.61876" x="142.72659" height="8.9989662" width="45.435822" @@ -408,37 +453,38 @@ style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.5;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:5.29166651px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="56.580025" - y="141.80258" - id="text4630"><tspan + style="font-style:normal;font-weight:normal;font-size:5.29167px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="57.109192" + y="134.39435" + id="text4630" + onclick="https://ladybird.dev/"><tspan sodipodi:role="line" - id="tspan4628" - x="56.580025" - y="141.80258" - style="font-size:5.29166651px;stroke-width:0.26458332">Firefox</tspan></text> + x="57.109192" + y="134.39435" + style="font-size:5.29167px;stroke-width:0.264583" + id="tspan22690">Ladybird</tspan></text> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:5.29166651px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + style="font-style:normal;font-weight:normal;font-size:5.29167px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" x="157.0732" - y="141.80258" + y="134.39435" id="text4634"><tspan sodipodi:role="line" id="tspan4632" x="157.0732" - y="141.80258" - style="font-size:5.29166651px;stroke-width:0.26458332">Emacs</tspan></text> + y="134.39435" + style="font-size:5.29167px;stroke-width:0.264583">Emacs</tspan></text> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:7.05555534px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" x="107.11687" - y="161.30249" + y="157.59837" id="text4638"><tspan sodipodi:role="line" id="tspan4636" x="107.11687" - y="161.30249" - style="font-size:7.05555534px;stroke-width:0.26458332">glibc</tspan></text> + y="157.59837" + style="font-size:7.05556px;stroke-width:0.264583">glibc</tspan></text> <flowRoot xml:space="preserve" id="flowRoot4642" @@ -450,16 +496,17 @@ height="48.487309" x="68.690376" y="330.83231" /></flowRegion><flowPara - id="flowPara4648"></flowPara></flowRoot> <text + id="flowPara4648" /></flowRoot> + <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="66.62764" - y="197.87538" + style="font-style:normal;font-weight:normal;font-size:10.5833px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="64.917892" + y="188.84837" id="text4652"><tspan sodipodi:role="line" - x="66.62764" - y="197.87538" - style="stroke-width:0.26458332" + x="64.917892" + y="188.84837" + style="stroke-width:0.264583" id="tspan4654">auth</tspan></text> <flowRoot xml:space="preserve" @@ -472,17 +519,18 @@ height="48.487309" x="354.69034" y="330.48755" /></flowRegion><flowPara - id="flowPara4664"></flowPara></flowRoot> <text + id="flowPara4664" /></flowRoot> + <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:7.05555534px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="137.01315" - y="197.15428" + style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="136.11226" + y="187.49091" id="text4668"><tspan sodipodi:role="line" id="tspan4666" - x="137.01315" - y="197.15428" - style="font-size:5.29166651px;stroke-width:0.26458332">other servers</tspan></text> + x="136.11226" + y="187.49091" + style="font-size:5.29167px;stroke-width:0.264583">other translators</tspan></text> <flowRoot xml:space="preserve" id="flowRoot4670" @@ -494,17 +542,18 @@ height="48.487309" x="69.639618" y="393.76147" /></flowRegion><flowPara - id="flowPara4676"></flowPara></flowRoot> <text + id="flowPara4676" /></flowRoot> + <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:8.81944466px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="66.104568" - y="213.70885" + style="font-style:normal;font-weight:normal;font-size:8.81944px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="53.057289" + y="205.39496" id="text4680"><tspan sodipodi:role="line" - id="tspan4678" - x="66.104568" - y="213.70885" - style="font-size:8.81944466px;stroke-width:0.26458332">pfinet</tspan></text> + x="53.057289" + y="205.39496" + style="font-size:8.81944px;stroke-width:0.264583" + id="tspan381">pfinet/lwip</tspan></text> <flowRoot xml:space="preserve" id="flowRoot4682" @@ -516,133 +565,222 @@ height="48.487309" x="354.69034" y="394.97464" /></flowRegion><flowPara - id="flowPara4688"></flowPara></flowRoot> <text + id="flowPara4688" /></flowRoot> + <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="136.1163" - y="215.54207" + style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="134.82208" + y="205.39278" id="text4692"><tspan sodipodi:role="line" id="tspan4690" - x="136.1163" - y="215.54207" - style="stroke-width:0.26458332">netdde</tspan></text> + x="134.82208" + y="205.39278" + style="font-size:7.05556px;stroke-width:0.264583">netdde/rump</tspan></text> <path - style="fill:none;stroke:#000000;stroke-width:0.60000002;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend)" - d="m 102.83915,211.57594 c 20.04525,-0.26727 20.04525,-0.26727 20.04525,-0.26727" + style="fill:none;stroke:#000000;stroke-width:0.6;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend)" + d="m 105.09742,203.60706 c 20.04525,-0.26727 20.04525,-0.26727 20.04525,-0.26727" id="path4694" inkscape:connector-curvature="0" /> - <path - style="fill:none;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker5380)" - d="m 177.67465,212.11048 c 2.85115,-0.0891 5.70203,-0.17819 8.18044,1.36615 2.47841,1.54434 4.58508,4.72277 5.43147,7.6181 0.84639,2.89533 0.43264,5.50799 -1.42247,8.23997 -1.85511,2.73199 -5.15105,5.58253 -10.09581,7.63174 -4.94476,2.04922 -11.53685,3.29637 -16.34782,3.60822 -4.81097,0.31185 -7.83988,-0.31175 -10.86914,-0.93542" - id="path5088" - inkscape:connector-curvature="0" - inkscape:path-effect="#path-effect5090" - inkscape:original-d="m 177.67465,212.11048 c 2.85114,-0.0894 5.70202,-0.17845 8.55263,-0.26727 2.10652,3.17756 4.21319,6.35599 6.31939,9.53438 -0.4135,2.6125 -0.82725,5.22516 -1.24127,7.83814 -3.29644,2.85093 -6.59238,5.70148 -9.88897,8.55262 -6.59295,1.2471 -13.18504,2.49425 -19.77795,3.74178 -3.02893,-0.62393 -6.05784,-1.24753 -9.08716,-1.87089" /> <rect - style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.45291704;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.452917;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect5454" width="44.099472" height="5.0439944" x="-136.45003" - y="141.85284" + y="134.44461" transform="scale(-1,1)" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:4.23333311px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="97.101242" - y="145.5336" + style="font-style:normal;font-weight:normal;font-size:4.23333px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="96.042908" + y="138.12537" id="text5458"><tspan sodipodi:role="line" id="tspan5456" - x="97.101242" - y="145.5336" - style="font-size:4.23333311px;stroke-width:0.26458332">Other Applications</tspan></text> + x="96.042908" + y="138.12537" + style="font-size:4.23333px;stroke-width:0.264583">Other Applications</tspan></text> <rect style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.5;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect5462" width="28.207415" height="6.7667108" x="42.982037" - y="158.83899" /> + y="155.13487" /> <rect - style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.52210951;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.52211;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect5466" width="30.858" height="6.7446012" x="74.375511" - y="158.85004" /> + y="155.14592" /> <rect - y="158.53973" + y="154.83562" x="125.90438" height="6.7446012" width="30.858" id="rect5468" - style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.52210951;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.52211;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> <rect - style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.52640831;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:0.526408;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" id="rect5470" width="31.388241" height="6.7403026" x="158.37404" - y="158.54189" /> + y="154.83777" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:5.29166651px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + style="font-style:normal;font-weight:normal;font-size:5.29167px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" x="48.359131" - y="163.97519" + y="160.27107" id="text5474"><tspan sodipodi:role="line" id="tspan5472" x="48.359131" - y="163.97519" - style="font-size:5.29166651px;stroke-width:0.26458332">send ()</tspan></text> + y="160.27107" + style="font-size:5.29167px;stroke-width:0.264583">send ()</tspan></text> <path - style="fill:none;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Lend)" - d="m 43.772571,140.48224 c -5.34513,-0.0891 -10.69052,-0.17818 -14.076045,1.82629 -3.385524,2.00446 -4.81089,6.10239 -4.409962,9.44351 0.400928,3.34112 2.627998,5.92452 4.766255,7.52818 2.138258,1.60366 4.187306,2.22728 6.236331,2.58362 2.049024,0.35633 4.09826,0.44542 6.147073,0.5345" + style="fill:none;stroke:#000000;stroke-width:0.504548;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 41.991837,133.26222 c -4.826748,-0.10046 -9.653728,-0.20092 -12.710917,2.0594 -3.057188,2.26032 -4.34432,6.88131 -3.982294,10.64885 0.362025,3.76754 2.373109,6.6807 4.303993,8.48906 1.930883,1.80837 3.78121,2.51159 5.631531,2.91342 1.850321,0.40183 3.700822,0.5023 5.550941,0.60275" id="path5480" inkscape:connector-curvature="0" inkscape:path-effect="#path-effect5482" - inkscape:original-d="m 43.772571,140.48224 c -5.345126,-0.0893 -10.690516,-0.17844 -16.036172,-0.26727 -1.425242,4.09807 -2.85061,8.196 -4.276312,12.2944 2.227684,2.58355 4.454757,5.16695 6.681738,7.75082 2.049348,0.62337 4.098398,1.24699 6.147199,1.87089 2.049163,0.0888 4.098398,0.17791 6.147199,0.26726" /> + inkscape:original-d="m 41.991837,133.26222 c -4.826741,-0.1008 -9.653722,-0.20122 -14.480943,-0.30138 -1.287018,4.62114 -2.57415,9.24213 -3.861584,13.86365 2.011637,2.91332 4.022723,5.82647 6.033725,8.74014 1.850598,0.70293 3.700925,1.40615 5.551029,2.10968 1.85043,0.10005 3.700925,0.20062 5.551027,0.30139" /> <path - style="fill:none;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker5138)" - d="m 62.481438,165.33831 c -0.712656,3.4742 -1.425374,6.9487 -3.563581,10.33429 -2.138207,3.38558 -5.701489,6.68162 -9.220664,9.79988 -3.519176,3.11827 -6.993414,6.058 -9.577165,9.57718 -2.583751,3.51918 -4.27632,7.61698 -3.830827,10.77992 0.445493,3.16294 3.028897,5.39001 5.835351,6.32543 2.806454,0.93542 5.835533,0.57906 8.864296,0.22274" + style="fill:none;stroke:#000000;stroke-width:0.399546;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker5138)" + d="m 54.936086,163.02099 c -0.508431,3.10959 -1.0169,6.21941 -2.54235,9.24964 -1.525451,3.03024 -4.067586,5.98034 -6.578249,8.77131 -2.510663,2.79098 -4.989255,5.42216 -6.832571,8.57198 -1.843317,3.14983 -3.050836,6.81753 -2.733034,9.64846 0.317801,2.83093 2.160868,4.82426 4.163091,5.66152 2.002222,0.83727 4.163238,0.51831 6.324021,0.19939" id="path5668" inkscape:connector-curvature="0" inkscape:path-effect="#path-effect5670" - inkscape:original-d="m 62.481438,165.33831 c -0.712454,3.47424 -1.425173,6.94874 -2.138156,10.42351 -3.563644,3.29635 -7.126923,6.59239 -10.690781,9.88898 -3.474507,2.93992 -6.948743,5.87966 -10.423512,8.81989 -1.692581,4.0982 -3.385149,8.196 -5.078121,12.2944 2.584071,2.22715 5.167475,4.45422 7.750816,6.68173 3.029295,-0.35662 6.058374,-0.71298 9.087164,-1.06907" /> + inkscape:original-d="m 54.936086,163.02099 c -0.508282,3.10961 -1.016752,6.21943 -1.525412,9.32949 -2.54239,2.95039 -5.084519,5.90049 -7.627062,8.85105 -2.478798,2.63138 -4.957401,5.26255 -7.436385,7.89419 -1.20753,3.66806 -2.415048,7.33576 -3.622855,11.004 1.843538,1.99338 3.686601,3.98671 5.529619,5.98043 2.161174,-0.31919 4.322192,-0.63813 6.483003,-0.95686" /> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:3.52777767px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" - x="78.036789" - y="163.15187" + style="font-style:normal;font-weight:normal;font-size:4.93889px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" + x="82.799294" + y="159.97691" id="text5686"><tspan sodipodi:role="line" - id="tspan5684" - x="78.036789" - y="163.15187" - style="font-size:3.52777767px;stroke-width:0.26458332">other functions</tspan></text> + x="82.799294" + y="159.97691" + style="font-size:4.93889px;stroke-width:0.264583" + id="tspan15664">write ()</tspan></text> <text id="text5690" - y="163.15187" - x="128.83672" - style="font-style:normal;font-weight:normal;font-size:3.52777767px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + y="159.97691" + x="132.54083" + style="font-style:normal;font-weight:normal;font-size:4.93889px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" xml:space="preserve"><tspan - style="font-size:3.52777767px;stroke-width:0.26458332" - y="163.15187" - x="128.83672" + style="font-size:4.93889px;stroke-width:0.264583" + y="159.97691" + x="132.54083" id="tspan5688" - sodipodi:role="line">other functions</tspan></text> + sodipodi:role="line">strlen ()</tspan></text> <text xml:space="preserve" - style="font-style:normal;font-weight:normal;font-size:3.52777767px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + style="font-style:normal;font-weight:normal;font-size:3.52778px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583" x="160.58667" - y="163.15187" + y="159.44775" id="text5694"><tspan sodipodi:role="line" id="tspan5692" x="160.58667" - y="163.15187" - style="font-size:3.52777767px;stroke-width:0.26458332">other functions</tspan></text> + y="159.44775" + style="font-size:3.52778px;stroke-width:0.264583">other functions</tspan></text> + <rect + style="fill:#000000;stroke-width:0.26863" + id="rect458" + width="169.04173" + height="1.4528459" + x="30.752951" + y="222.75043" /> + <text + xml:space="preserve" + style="font-size:3.175px;fill:#000000;stroke-width:0.264583" + x="62.755424" + y="235.7558" + id="text570"><tspan + sodipodi:role="line" + id="tspan568" + style="stroke-width:0.264583" + x="62.755424" + y="235.7558" /></text> + <text + xml:space="preserve" + transform="matrix(0.39968629,0,0,0.42519352,-66.453973,31.427863)" + id="text650" + style="font-size:12px;white-space:pre;shape-inside:url(#rect652);display:inline;fill:#000000"><tspan + x="277.88867" + y="445.02202" + id="tspan21500">userspace (root) +</tspan></text> + <rect + style="fill:#87cdde;stroke:#000000;stroke-width:0.245903;stroke-opacity:1" + id="rect827" + width="46.805042" + height="10.69946" + x="134.07603" + y="241.63672" + inkscape:highlight-color="#70ee3b" /> + <text + xml:space="preserve" + transform="matrix(0.26458333,0,0,0.26458333,24.853071,129.0762)" + id="text2104" + style="font-size:18.6667px;white-space:pre;shape-inside:url(#rect2106);display:inline;fill:#87decd;stroke:#000000;stroke-opacity:1"><tspan + x="433.82617" + y="452.57812" + id="tspan21504"><tspan + style="fill:#0b0b28" + id="tspan21502">network board</tspan></tspan></text> + <text + xml:space="preserve" + transform="matrix(0.19700851,0,0,0.20398568,49.42454,125.9867)" + id="text5136" + style="font-size:18.6667px;white-space:pre;shape-inside:url(#rect5138);display:inline;fill:#ffaacc;stroke:#000000;stroke-opacity:1"><tspan + x="71.042969" + y="519.37499" + id="tspan21508"><tspan + style="fill:#000000" + id="tspan21506">kernel space</tspan></tspan></text> + <path + style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.366964;stroke-opacity:1;marker-end:url(#marker5632)" + d="m 183.96162,204.2343 c 6.91051,4.42661 11.58258,12.17546 12.27167,20.35319 0.69511,8.24915 -2.68264,16.74417 -8.8524,22.26375" + id="path5504" + inkscape:path-effect="#path-effect5506" + inkscape:original-d="m 183.96162,204.2343 c 4.09081,6.77337 8.18138,13.56931 12.27167,20.35319 4.0903,6.78387 -5.90134,14.84301 -8.8524,22.26375" /> + <path + style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:1.16715;stroke-dasharray:9.33723, 1.16715;stroke-dashoffset:0;stroke-opacity:1" + d="M 30.784693,170.07574 H 200.39486" + id="path12753" /> + <text + xml:space="preserve" + transform="matrix(0.26458333,0,0,0.26458333,37.315126,89.65673)" + id="text15062" + style="font-size:16px;white-space:pre;shape-inside:url(#rect15064);display:inline;fill:#24221c;fill-opacity:0;stroke:#000000;stroke-width:0.377953;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1"><tspan + x="70.617188" + y="219.17324" + id="tspan21512"><tspan + style="fill:#000000;fill-opacity:1" + id="tspan21510">user: bob</tspan></tspan></text> + <circle + id="path2555" + style="fill:#000000;stroke:none;stroke-width:0.264583" + cx="30.373119" + cy="238.95235" + r="0.14999999" /> + <path + style="fill:#ffffff;fill-opacity:1;stroke:#000000;stroke-width:1;stroke-dasharray:none" + d="M 30.279233,239.07937 H 200.39252" + id="path2557" /> + <text + xml:space="preserve" + transform="matrix(0.26458333,0,0,0.26458333,22.49845,107.64845)" + id="text15993" + style="font-size:18.6667px;white-space:pre;shape-inside:url(#rect15995);fill:#ffffff;fill-opacity:1;stroke:#000000;stroke-width:0.377953;stroke-dasharray:none"><tspan + x="153.79883" + y="529.64843" + id="tspan21516"><tspan + style="fill:#000000" + id="tspan21514">hardware</tspan></tspan></text> </g> </svg> diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index 1a06897b..3da5e558 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -1163,7 +1163,7 @@ License|/fdl]]."]]"""]] ## IRC, freenode, #hurd, 2012-09-23 <braunr> tschwinge: i committed the last hurd pthread change, - http://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=master-pthreads + https://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=master-pthreads <braunr> tschwinge: please tell me if you consider it ok for merging @@ -1646,13 +1646,13 @@ See also [[open_issues/libpthread/fix_have_kernel_resources]]. does it right <braunr> in the io_select_timeout branch <braunr> see - http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/libthreads/cancel-cond.c?h=rbraun/select_timeout + https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/libthreads/cancel-cond.c?h=rbraun/select_timeout for example * pinotree looks <braunr> what matters is the msg_delivered member used to synchronize sleeper and waker <braunr> the waker code is in - http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/libthreads/cprocs.c?h=rbraun/select_timeout + https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/libthreads/cprocs.c?h=rbraun/select_timeout <pinotree> never seen cthreads' code before :) <braunr> soon you shouldn't have any more reason to :p <pinotree> ah, so basically the cthread version of the pthread cleanup diff --git a/open_issues/mach-defpager_swap.mdwn b/open_issues/mach-defpager_swap.mdwn index 6e4dc088..78290d71 100644 --- a/open_issues/mach-defpager_swap.mdwn +++ b/open_issues/mach-defpager_swap.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012,2025 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -13,6 +13,22 @@ License|/fdl]]."]]"""]] [[!toc]] +# Out of memory situations + +GNU Mach currently does not know how much room mach-defpager has. When it is +full, mach-defpager will start not being able to write data and thus just +lose it. The corresponding object will trigger SIGBUS on access to lost data +(hopefully killing the corresponding process, thus releasing memory). + +GNU Mach should be aware of how much is left, to better manage paging out, +possibly throttling some processes. Hopefully ext2fs will be able to page out +dirty data and release memory. With overcommit control we can then avoid one +process filling the whole memory+swap with anonymous data. If several processes +cooperate in sucking memory, we'd have to OOM some of them. + +E.g. `vm_page_evict` would know not to try to evict an internal page, and just +wait for the external pages to get paged out and get released. + # IRC, OFTC, #debian-hurd, 2012-06-16 <lifeng> I allocated a 5GB partition as swap, but hurd only found 1GB diff --git a/open_issues/nice_changes_priority_of_parent_shell.mdwn b/open_issues/nanosecond-precision_clock.mdwn index a8a08f90..65690e96 100644 --- a/open_issues/nice_changes_priority_of_parent_shell.mdwn +++ b/open_issues/nanosecond-precision_clock.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013, 2014, 2025 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,8 +9,14 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_gnumach open_issue_glibc]] +[[!meta title="a nanosecond-precision clock"]] - * [[!debbug 44039]] +[[!tag open_issue_glibc open_issue_gnumach]] + +What about adding a nanosecond-precision clock, too? --[[tschwinge]] + +The y2038 64b interfaces and the mapped time already support it, but the +granularity is still the clock tick period. + +# Candidate for [[vDSO]] code? - * Also see [[nice_vs_mach_thread_priorities]]. diff --git a/open_issues/nice_vs_mach_thread_priorities.mdwn b/open_issues/nice_vs_mach_thread_priorities.mdwn deleted file mode 100644 index 1f4c6ab8..00000000 --- a/open_issues/nice_vs_mach_thread_priorities.mdwn +++ /dev/null @@ -1,429 +0,0 @@ -[[!meta copyright="Copyright © 2010, 2012, 2013 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_gnumach open_issue_glibc]] - -This issue has been known for some time, due to coreutils' testsuite choking -when testing *nice*: [[!debbug 190581]]. - -There has been older discussion about this, too, but this is not yet captured -here. - - -# IRC, freenode, #hurd, 2010-08 - - <pochu> I'm reading Mach and POSIX documentation to understand the priorities/nice problems - <pochu> antrik said it would be better to reimplement everything instead of - fixing the current Mach interfaces, though I'm not sure about that yet - <youpi> uh, so he changed his mind? - <pochu> it seems POSIX doesn't say nice values should be -20..20, but - 0..(2*NZERO - 1) - <youpi> he said we could just change the max priority value and be done - with it :) - <pochu> so we can probably define NZERO to 16 to match the Mach range of - 0..31 - <youpi> s/said/had said previously/ - <antrik> youpi: POSIX is actually fucked up regarding the definition of - nice values - <antrik> or at least the version I checked was - <pochu> antrik: why? this says the range is [0,{NZERO}*2-1], so we can just - set NZERO to 16 AFAICS: - http://www.opengroup.org/onlinepubs/9699919799/functions/getpriority.html - <antrik> it talkes about NZERO and all; making it *look* like this could be - defined arbitrarily... but in other places, it's clear that the standard - 40 level range is always assumed - <antrik> anyways, I totally see no point in deviating from other systems in - this regard. it can only cause problems, and gives us no benefits - <cfhammar> it says NZERO should be at least 20 iirc - <youpi> agreed - <antrik> I don't remember the details; it's been a while since I looked at - this - <antrik> youpi: changing the number of levels is only part of the - issue. I'm not sure why I didn't mention it initially when we discussed - this - <antrik> youpi: I already concluded years ago that it's not possible to - implement nice levels correctly with the current Mach interfaces in a - sane fashion - <antrik> (it's probably possible, but only with a stupid hack like setting - all the thread priorities one by one) - <antrik> youpi: also, last time we discussed this, I checked how the nice - stuff works currently on Hurd; and concluded that it's so utterly broken, - that there is no point in trying to preserve *any* compatibility. I think - we can safely throw away any handling that is alread there, and do it - over from scratch in the most straightforward fashion - <pochu> antrik: I've thought about setting NZERO to 16 and doing exactly - what you've just said to be a hack (setting all the thread priorities one - by one) - <pochu> but there seems to be consensus that that's undesirable... - <pochu> indeed, POSIX says NZERO should be at least 20 - <antrik> pochu: BTW, I forgot to say: I'm not sure you appreciate the - complexity of setting the thread max priorities individually - <pochu> antrik: I don't. would it be too complex? I imagined it would be a - simple loop :) - <antrik> pochu: in order to prevent race conditions, you have to stop all - other threads before obtaining the list of threads, and continue them - after setting the priority for each - <antrik> I don't even know whether it can be done without interfering with - other thread handling... in which case it gets really really ugly - <pochu> antrik: btw I'm looking at [gnumach]/kern/thread.[ch], removing the - priority stuff as appropriate, and will change the tasks code later - <antrik> it seems to me that using a more suitable kernel interface will - not only be more elegant, but quite possibly actually easier to - implement... - <pochu> antrik: apparently it's not that hard to change the priority for - all threads in a task, see task_priority() in gnumach/kern/task.c - <pochu> it looks like the nice test failures are mostly because of the not - 1:1 mapping between nice values and Mach priorities - <marcusb> "Set priority of task; used only for newly created threads." - <marcusb> there is a reason I didn't fix nice 8 years ago - <marcusb> ah there is a change_threads option - <pochu> marcusb: I'm not sure that comment is correct. that syscall is used - by setpriority() - <marcusb> yeah - <marcusb> I didn't read further, where it explains the change_threads - options - <marcusb> I was shooting before asking questions :) - <marcusb> pochu: although there are some bad interactions if max_priorities - are set per thread - <antrik> pochu: maybe we are talking past each other. my point was not that - it's hard to do in the kernel. I was just saying that it would be painful - to do from userspace with the current kernel interface - <pochu> antrik: you could still use that interface in user space, couldn't - you? or maybe I'm misunderstanding... - <pochu> cfhammar, antrik: current patch: - http://emilio.pozuelo.org/~deb/gnumach.patch, main issue is probably what - to do with high-priority threads. are there cases where there should be a - thread with a high priority but the task's priority shouldn't be high? - e.g. what to do with kernel_thread() in [gnumach]/kern/thread.c - <pochu> i.e. if tasks have a max_priority, then threads shouldn't have a - higher priority, but then either we raise the task's max_priority if we - need a high-prio thread, or we treat them specially (e.g. new field in - struct thread), or maybe it's a non-issue because in such cases, all the - task is high-prio? - <pochu> also I wonder whether I can kill the processor set's - max_priority. It seems totally unused (I've checked gnumach, hurd and - glibc) - <pochu> (that would simplify the priority handling) - <cfhammar> pochu: btw what does your patch do? i can't remember what was - decided - <pochu> cfhammar: it moves the max_priority from the thread to the task, so - raising/lowering it has effect on all of its threads - <pochu> it also increases the number of run queues (and thus that of - priority levels) from 32 to 40 so we can have a 1:1 mapping with nice - values - <pochu> cfhammar: btw don't do a full review yet, just a quick look would - be fine for now - <neal> why not do priorities from 0 to 159 - <neal> then both ranges can be scaled - <neal> without loss of precision - <pochu> neal: there would be from Mach to nice priorities, e.g. a task with - a priority of 2 another with 3 would have the same niceness, though their - priority isn't really the same - <neal> pochu: sure - <neal> pochu: but any posix priority would map to a current mach priority - and back - <neal> sorry, that's not true - <neal> a posix priority would map to a new mach priority and bach - <neal> and a current mach priority would map to a new mach priority and - back - <neal> which is I think more desirable than changing to 40 priority levels - <pochu> neal> and a current mach priority would map to a new mach priority - and back <- why should we care about this? - <neal> to be compatible with existing mach code - <neal> why gratutiously break existing interfaces? - <pochu> they would break anyway, wouldn't them? i.e. if you do - task_set_priority(..., 20), you can't know if the caller is assuming old - or new priorities (to leave it as 20 or as 100) - <neal> you add a new interface - <neal> you should avoid changing the semantics of existing interfaces as - much as possible - <pochu> ok, and deprecate the old ones I guess - <neal> following that rule, priorities only break if someone does - task_set_priority_new(..., X) and task_get_priority () - <neal> there are other users of Mach - <neal> I'd add a configure check for the new interface - <neal> alternatively, you can check at run time - <pochu> well if you _set_priority_new(), you should _get_priority_new() :) - <neal> it's not always possible - <pochu> other users of GNU Mach? - <neal> you are assuming you have complete control of all the code - <neal> this is usually not the case - <neal> no, other users of Mach - <neal> even apple didn't gratuitously break Mach - <neal> in fact, it may make sense to see how apple handles this problem - <pochu> hmm, I hadn't thought about that - <pochu> the other thing I don't understand is: "I'd add a configure check - for the new interface". a configure check where? in Mach's configure? - that doesn't make sense to me - <neal> any users of the interface - <pochu> ok so in clients, e.g. glibc & hurd - <neal> yes. - <antrik> neal: I'm not sure we are winning anything by keeping - compatibility with other users of Mach... - <antrik> neal: we *know* that to make Hurd work really well, we have to do - major changes sooner or later. we can just as well start now IMHO - <antrik> keeping compatibility just seems like extra effort without any - benefit for us - <guillem> just OOC have all other Mach forks, preserved full compatibility? - <neal> guillem: Darwin is pretty compatible, as I understand it - <antrik> pochu: the fundamental approach of changing the task_priority - interface to serve as a max priority, and to drop the notion of max - priorities from threads, looks fine - <antrik> pochu: I'm not sure about the thread priority handling - <antrik> I don't know how thread priorities are supposed to work in chreads - and/or pthread - <antrik> I can only *guess* that they assume a two-stage scheduling - process, where the kernel first decides what process to run; and only - later which thread in a process... - <antrik> if that's indeed the case, I don't think it's even possible to - implement with the current Mach scheduler - <antrik> I guess we could work with relative thread priorities if we really - want: always have the highest-priority thread run with the task's max - priority, and lower the priorities of the other threads accordingly - <antrik> however, before engaging into this, I think you should better - check whether any of the code in Hurd or glibc actually uses thread - priorities at all. my guess is that it doesn't - <antrik> I think we could get away with stubbing out thread priority - handling alltogether for now, and just use the task priority for all - threads - <antrik> I agree BTW that it would be useful to check how Darwin handles - this - <pochu> btw do you know where to download the OS X kernel source? I found - something called xnu, but I?m not sure that's it - <antrik> pochu: yeah, that's it - <antrik> Darwin is the UNIX core of OS X, and Xnu is the actual kernel... - <pochu> hmm, so they have both a task.priority and a task.max_priority - <neal> pochu: thoughts? - <pochu> neal: they have a priority and a max_priority in the task and in - the threads, new threads inherit it from its parent task - <pochu> then they have a task_priority(task, priority, max_priority) that - can change a task's priorities, and it also changes it for all its - threads - <neal> how does the global run queue work? - <pochu> and they have 128 run queues, no idea if there's a special reason - for that number - <pochu> neal: sorry, what do you mean? - <neal> I don't understand the point of the max_priority parameter - <pochu> neal: and I don't understand the point of the (base) priority ;) - <pochu> the max_priority is just that, the maximum priority of a thread, - which can be lowered, but can't exceed the max one - <pochu> the (base) priority, I don't understand what it does, though I - haven't looked too hard. maybe it's the one a thread starts at, and must - be <= max_priority - <antrik> pochu: it's clearly documented in the manual, as well as in the - code your initial patch changes... - <antrik> or do you mean the meaning is different in Darwin?... - <pochu> I was speaking of Darwin, though maybe it's the same as you say - <antrik> I would assume it's the same. I don't think there would be any - point in having the base vs. max priority distinction at all, except to - stay in line with standard Mach... - <antrik> at least I can't see a point in the base priority semantics for - use in POSIX systems... - <pochu> right, it would make sense to always have priority == max_priority - ... - <pochu> neal: so max_priority is that maximum priority, and priority is the - one used to calculate the scheduled priority, and can be raised and - lowered by the user without giving special permissions as long as he - doesn't raise it above max_priority - <pochu> well this would allow a user to lower a process' priority, and - raise it again later, though that may not be allowed by POSIX, so then we - would want to have max_priority == priority (or get rid of one of them if - possible and backwards compatible) - <antrik> pochu: right, that's what I think too - <antrik> BTW, did I bring up handling of thread priorities? I know that I - meant to, but I don't remember whether I actually did... - <pochu> antrik: you told me it'd be ok to just get rid of them for now - <pochu> so I'm more thinking of fixing max_priority and (base) priority and - leaving thread's scheduling priority as it currently is - <pochu> s/so/though/ - <antrik> pochu: well, my fear is that keeping the thread priority handling - as ist while changing task priority handling would complicate the - changes, while giving us no real benefit... - <antrik> though looking at what Darwin did there should give you an idea - what it involves exactly... - <pochu> antrik: what would you propose, keeping sched_priority == - max_priority ? - <pochu> s/keeping/making/ - <antrik> yes, if that means what I think it does ;-) - <antrik> and keeping the priority of all threads equal to the task priority - for now - <antrik> of course this only makes sense if changing it like this is - actually simpler than extending the current handling... - <antrik> again, I can't judge this without actually knowing the code in - question. looking at Darwin should give you an idea... - <pochu> I think leaving it as is, making it work with the task's - max_priority changes would be easier - <antrik> perhaps I'm totally overestimating the amount of changes required - to do what Darwin does - <antrik> OTOH, carrying around dead code isn't exactly helping the - maintainability and efficiency of gnumach... - <antrik> so I'm a bit ambivalent on this - <antrik> should we go for minimal changes here, or use this occasion to - simplify things?... - <antrik> I guess it would be good to bring this up on the ML - <cfhammar> in the context of gsoc i'd say minimal changes - <pochu> there's also neal's point on keeping backwards compatibility as - much as possible - <neal> my point was not backwards compatibility at all costs - <antrik> I'm still not convinced this is a valid point :-) - <neal> but to not gratutiously break things - <antrik> neal: well, I never suggested breaking things just because we - can... I only suggested breaking things to make the code and interface - simpler :-) - <antrik> I do not insist on it though - <neal> at that time, we did not know how Mac did it - <antrik> I only think it would be good to get into a habit that Mach - interfaces are not sacred... - <neal> and, I also had a proposal, which I think is not difficult to - implement given the existing patch - <antrik> but as I said, I do not feel strongly about this. if people feel - more confident about a minimal change, I'm fine with that :-) - <antrik> neal: err... IIRC your proposal was only about the number of nice - levels? we are discussing the interface change necessary to implement - POSIX semantics properly - <antrik> or am I misremembering? - <pochu> antrik: he argues that with that number of nice levels, we could - keep backwards compatibility for the 0..31 levels, and for 0..39 for - POSIX compatibility - <antrik> pochu: yes, I remember that part - <neal> antrik : My suggestion was: raise the number of nice levels to 160 - and introduce a new interface which uses those. Adjust the old interface - to space by 160/32 - <antrik> neal: I think I said it before: the problem is not *only* in the - number of priority levels. the semantics are also wrong. which is why - Darwin added a max_priority for tasks - <neal> what do you mean the semantics are wrong? - <neal> I apologize if you already explained this. - <antrik> hm... I explained it at some point, but I guess you were not - present at that conversation - <neal> I got disconnected recently so I likely don't have it in backlog. - <antrik> in POSIX, any process can lower its priority; while only - privileged processes can raise it - <antrik> Mach distinguishes between "current" and "max" priority for - threads: "max" behaves like POSIX; while "current" can be raised or - lowered at will, as long as it stays below "max" - <antrik> for tasks, there is only a "current" priority - <antrik> (which applies to newly created threads, and optionally can be set - for all current threads while changing the task priority) - <antrik> glibc currently uses the existing task priorities, which leads to - *completely* broken semantics - <antrik> instead, we need something like a max task priority -- which is - exactly what Darwin added - <neal> yes - <antrik> (the "current" task priority is useless for POSIX semantics as far - as I can tell; and regarding thread priorities, I doubt we actually use - them at all?...) - <cfhammar> where does a new thread get its initial max_priority from? - <antrik> cfhammar: from the creator thread IIRC - <pochu> yes - - -## IRC, freenode, #hurd, 2010-08-12 - - <pochu> my plan is to change the number of priority levels and the - threads/tasks priority handling, then add new RPCs to play with them and - make the old ones stay compatible, then make glibc use the new RPCs - - -# IRC, freenode, #hurd, 2012-12-29 - - <braunr> and, for a reason that i can't understand, there are less - priorities than nice levels, despite the fact mach was designed to run - unix systems on top of it .. - <youpi> btw, didn't we have a plan to increase that number? - <braunr> i have no idea - <braunr> but we should :) - <youpi> I remember some discussion about it on the list - - -## IRC, freenode, #hurd, 2012-12-31 - - <youpi> braunr: btw, we *do* have fixed the nice granularity - <youpi> +#define MACH_PRIORITY_TO_NICE(prio) ((prio) - 20) - <youpi> in the debian package at least - <youpi> ah, no - <youpi> it's not applied yet - <youpi> so I have the patch under hand, just not applied :) - <braunr> but that's a simple shift - <braunr> the real problem is that there aren't as many mach priorities as - there are nice levels - <youpi> that's not really a problem - <youpi> we can raise that in the kernel - <youpi> the problem is the change from shifted to unshifted - <youpi> that brings odd nice values during the upgrade - <braunr> ok - <braunr> i hope the scheduler code isn't allergic to more priorities :) - - -## IRC, freenode, #hurd, 2013-01-02 - - <braunr> pochu: i see you were working on nice levels and scheduling code - some time ago - <braunr> pochu: anything new since then ? - <pochu> braunr: nope - <braunr> pochu: were you preparing it for the gsoc ? - <pochu> braunr: can't remember right now, either that or to fix a ftbfs in - debian - <youpi> iirc it's coreutils which wants proper nice levels - - -# IRC, OFTC, #debian-hurd, 2013-03-04 - - <Steap> Is it not possible to set the priority of a process to 1 ? - <Steap> these macros: - <Steap> #define MACH_PRIORITY_TO_NICE(prio) (2 * ((prio) - 12)) - <Steap> #define NICE_TO_MACH_PRIORITY(nice) (12 + ((nice) / 2)) - <Steap> are used in the setpriority() implementation of Hurd - <Steap> so setting a process' priority to 1 is just like setting it to 0 - <youpi> Steap: that has already been discussed to drop the *2 - <youpi> the issue is mach not supporting enough sched levels - <youpi> can be fixed, of course - <youpi> just nobody did yet - -GNU Mach commit 6a234201081156e6d5742e7eeabb68418b518fad (and commit -6fe36b76f7983ec9a2e8a70420e431d54252442e). - - -## IRC, OFTC, #debian-hurd, 2013-03-07 - - <braunr> youpi: btw, why did you increase the number of priorites to 50 ? - <youpi> for the nice levels - <braunr> and probably something more, there are only 40 nice levels - <youpi> yes, the current computation leaves some margin - <youpi> so I preferred to keep a margin too - <braunr> ok - <youpi> e.g. for the idle thread, etc. - <braunr> or interrupt threads - <youpi> yep - <braunr> i see the point, thanks - <tschwinge> Is the number of 40 specified by POSIX (or whatever) or is that - a Linuxism? - <braunr> good question - <braunr> posix is weird when it comes to such old unixisms - <braunr> there is a NZERO value, but i don't remember how it's specified - <youpi> it's at least 20 - <tschwinge> (I don't object to the change; just wondered. And if practice, - you probably wouldn't really need more than a handful. But if that - change (plus some follow-up in glibc (?) improves something while not - adding a lot of overhead, then that's entirely fine, of course.) - <braunr> "A maximum nice value of 2*{NZERO}-1 and a minimum nice value of 0 - shall be imposed by the system" - <braunr> NZERO being 20 by default - <youpi> and 20 is the minimum for NZERO too - <braunr> hm, not the default, the minimul - <braunr> minimum - <braunr> yes that's it - <braunr> ok so it's actually well specified - <tschwinge> Aha, I see (just read it, too). So before that change we - simply couldn't satisfy the POSIX requirement of (minimum) NZERO = 20, - and allowing for step-1 increments. Alright. - <youpi> yep - <youpi> thus failing in coreutils testsuite diff --git a/open_issues/pci_arbiter.mdwn b/open_issues/pci_arbiter.mdwn index 7fdb4323..4efb79f3 100644 --- a/open_issues/pci_arbiter.mdwn +++ b/open_issues/pci_arbiter.mdwn @@ -25,11 +25,11 @@ sequential: gnumach, netdde, and then Xorg. The PCI Arbiter will eventually allo concurrent access to the PCI config space. The Hurd now has a PCI Arbiter, but it could use some more polishing. You can find -its TODO file [[here|http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/pci-arbiter/TODO]]. +its TODO file [[here|https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/pci-arbiter/TODO]]. Samuel also gave a presentation explaining some of the awesome possibilities that the PCI Arbiter can provide. You can watch his fosdem talk -[[here|https://archive.fosdem.org/2018/schedule/event/microkernel_hurd_pci_arbiter/]] +[[here|https://archive.fosdem.org/2018/schedule/event/microkernel_hurd_pci_arbiter/]]. The end goal is to allow different userland drivers to access PCI devices concurrently, while paving the way for fine-grain per-user, per-session, etc. permission management diff --git a/open_issues/problematic_packages.mdwn b/open_issues/problematic_packages.mdwn index 8fe06495..22054e6e 100644 --- a/open_issues/problematic_packages.mdwn +++ b/open_issues/problematic_packages.mdwn @@ -26,11 +26,6 @@ This page lists the few packages whose build makes the Debian buildd box crash a * rsyslog -* ext2fs gets stuck - - * emacs24 - * emacs25 - * loops and eats memory * pygobject diff --git a/open_issues/resource_management_problems.mdwn b/open_issues/resource_management_problems.mdwn index daf97954..d1a29a00 100644 --- a/open_issues/resource_management_problems.mdwn +++ b/open_issues/resource_management_problems.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2013 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2013, 2024 Free +Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -18,7 +18,35 @@ Mach can't do a good job at resource management, as it doesn't have enough information how resources are used: which data is important and which is discardable, for example. -These issues are what Neal Walfield is working on with his new kernel +<!-- the following 3 paragraphs comes from this email: + lists.gnu.org/archive/html/bug-hurd/2009-07/msg00084.html --> + +This is the fundamental failing of the Mach/Hurd architecture. The +Hurd operates via many server to client relationships, in which +servers request resources on behalf of their clients. For example at +any given time, `extfs` could have many different clients (emacs, vim, +git etc.) requesting data, creating files, deleting files, re-naming +files, etc. Suppose one rogue client out of 50 is continually +requesting increasingly more memory, which is exhasting the machine's +resources. As far as Mach knows, `ext2fs` is wasting RAM. It doesn't +know that one `ext2fs`' client program is at fault. There is no way +for Mach to fix this, since it should not kill `ext2fs`, and it cannot +know which `ext2fs` client to kill. + +This server/client architecture is a problem that exists elsewhere. A +good example is `X`. Firefox might allocate a lot of pixmaps, which +causes `X` to use more memory. Linux actually used to kill X, because +of this several years ago. + +This problem is much worse on a multiserver system, because we have +many server/client relationships. A simple fix that would limit these +issues is to introduce fixed limits on various kinds of resource +usage. A proper fix requires a way to attribute all resource usage to +the clients -- either by avoiding server-side allocation or by keeping +track of who is requesting resources. Both of these changes requires +lots of changes to low-level code. + +These issues are what Neal Walfield explored with his kernel: [[microkernel/viengoos]]. diff --git a/open_issues/rpc_stub_generator.mdwn b/open_issues/rpc_stub_generator.mdwn index d4622d67..26511d3b 100644 --- a/open_issues/rpc_stub_generator.mdwn +++ b/open_issues/rpc_stub_generator.mdwn @@ -122,7 +122,7 @@ License|/fdl]]."]]"""]] <neal> the first are a burden <neal> the latter is a pain <neal> - http://git.savannah.gnu.org/gitweb/?p=hurd/viengoos.git;a=blob;f=libviengoos/viengoos/rpc.h;h=721768358a0299637fb79f226aea6a304571da85;hb=refs/heads/viengoos-on-bare-metal + https://git.savannah.gnu.org/gitweb/?p=hurd/viengoos.git;a=blob;f=libviengoos/viengoos/rpc.h;h=721768358a0299637fb79f226aea6a304571da85;hb=refs/heads/viengoos-on-bare-metal <neal> in the same directory, you there are headers that use it <braunr> neal: cf. http://genode.org/documentation/release-notes/11.05 <braunr> tschwinge: why do you recommend an IDL ? diff --git a/open_issues/runit.mdwn b/open_issues/runit.mdwn index 659b81ea..cda495ae 100644 --- a/open_issues/runit.mdwn +++ b/open_issues/runit.mdwn @@ -9,42 +9,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_porting]] +[[!tag open_issue]] -The `runit` package doesn't work, even its test suite doesn't finish. +In March of 2025, João got the Hurd to use +[[runit instead of sysvinit|https://lists.debian.org/debian-hurd/2025/03/msg00023.html]]. +He has so far been unsuccessful to start the [[hurd-console|hurd/console]]. -[[Thomas_Schwinge|tschwinge]] once was having a look at that, but this very -report is just from his memory, and his memory is dim... The problem *might* -either be a time stamping issue (which might be fixed by now) or it *might* be -the `select` call failing issue we're seeing from time to time. Or something -else. - -[[Harish Badrinath|harishbadrinath]] -Originally answered by Samuel Thibault: -> 120->proc_dostop_request ( 138) = 0 -> -> </snip> - -Usual issue with rpctrace: it does not support fork(). - - I've checked a backtrace in gdb, got this: - - 0x0105af6c in mach_msg_trap () - at /build/eglibc-jWVnRE/eglibc-2.13/build-tree/hurd-i386-libc/mach/mach_msg_trap.S:2 - 1 0x0105b769 in __mach_msg (msg=0x1024af8, option=258, send_size=0, rcv_size=40, rcv_name=140, - timeout=1000020, notify=0) at msg.c:110 - 2 0x01062251 in _hurd_select (nfds=2, pollfds=0x1024dc0, readfds=0x0, writefds=0x0, exceptfds=0x0, - timeout=0x1024bbc, sigmask=0x0) at hurdselect.c:324 - 3 0x0114427b in __poll (fds=0x1024dc0, nfds=2, timeout=1000020) at ../sysdeps/mach/hurd/poll.c:48 - 4 0x0804b770 in iopause (x=0x1024dc0, len=2, deadline=0x1024dd8, stamp=0x1024de8) at iopause.c:29 - 5 0x08048efc in main (argc=2, argv=0x1024e94) at runsv.c:543 - - and main() shows up as: - - sig_unblock(sig_term); - sig_unblock(sig_child); - -> iopause(x, 2 +haslog, &deadline, &now); - sig_block(sig_term); - sig_block(sig_child); - -So it simply looks like the known "signals don't interrupt select" bug. +The Hurd project could also use the [[Shepherd|open_issues/shepherd]] +as another alternative init system. diff --git a/open_issues/running_rump_for_slash.mdwn b/open_issues/running_rump_for_slash.mdwn index 52c97b5d..0740993e 100644 --- a/open_issues/running_rump_for_slash.mdwn +++ b/open_issues/running_rump_for_slash.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] +## Samuel's [[email|https://lists.gnu.org/archive/html/bug-hurd/2019-12/msg00002.html]] + I have been thinking about how to get rump running for the / filesystem. Looking at how things go between ext2fs and exec: in grub.cfg we have @@ -51,3 +53,20 @@ and we'd make `pci-arbiter`'s initialization use `task_set_special_port` to set the `TASK_BOOTSTRAP_PORT` special port of rump to a send right to pci-arbiter and resume it. When `libpciaccess` sees that this port is set, it would use it instead of looking up `/server/bus/pci`. + +## Damien's [[follow up email|https://lists.gnu.org/archive/html/bug-hurd/2020-04/msg00032.html]] + +In my own words, the changes that are needed: + +1. libpciaccess needs to check if pci-arbiter's `task_bootstrap_port` is set +and if it is, use it instead of `/servers/bus/pci` + +2. pci-arbiter needs to call `task_set_special_port` on rump's `TASK_BOOTSTRAP_PORT` +if `--disk-server-task` flag is detected. + +3. rumpdisk needs to call `task_set_special_port` on ext2fs's `TASK_BOOTSTRAP_PORT` +if `--fs-server-task` flag is detected. + +4. ext2fs will just work in this configuration + + diff --git a/open_issues/select.mdwn b/open_issues/select.mdwn index caecc437..19468d90 100644 --- a/open_issues/select.mdwn +++ b/open_issues/select.mdwn @@ -1103,7 +1103,7 @@ IRC, unknown channel, unknown date: <braunr> unfortunately, my idea alone isn't enough <braunr> for those interested in the problem, i've updated the analysis in my last commit - (http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout&id=40fe717ba9093c0c893d9ea44673e46a6f9e0c7d) + (https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout&id=40fe717ba9093c0c893d9ea44673e46a6f9e0c7d) #### IRC, freenode, #hurd, 2012-08-01 @@ -1660,7 +1660,7 @@ IRC, unknown channel, unknown date: #### IRC, freenode, #hurd, 2012-12-17 <braunr> tschwinge: - http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/select_timeout_for_one_fd + https://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/select_timeout_for_one_fd <braunr> gnu_srs: talking about that, can you explain : <braunr> "- The pure delay case is much faster now, making it necessary to <braunr> introduce a delay of 1 msec when the timeout parameter is set to diff --git a/open_issues/serial_console.mdwn b/open_issues/serial_console.mdwn index 827fd211..3100d60c 100644 --- a/open_issues/serial_console.mdwn +++ b/open_issues/serial_console.mdwn @@ -67,7 +67,7 @@ License|/fdl]]."]]"""]] <braunr> we definitely want it to work with 8N1 <gg0> never had problems with _virtual_ serial consoles <gg0> never = during last 2 years = since - http://git.savannah.gnu.org/gitweb/?p=hurd/gnumach.git;a=commitdiff;h=2a603e88f86bee88e013c2451eacf076fbcaed81 + https://git.savannah.gnu.org/gitweb/?p=hurd/gnumach.git;a=commitdiff;h=2a603e88f86bee88e013c2451eacf076fbcaed81 <gg0> but i don't think i was on real hardware at that time diff --git a/open_issues/serverbootv2.mdwn b/open_issues/serverbootv2.mdwn new file mode 100644 index 00000000..21a2f3ec --- /dev/null +++ b/open_issues/serverbootv2.mdwn @@ -0,0 +1,899 @@ +[[!meta copyright="Copyright © 2024 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# ServerBootV2 RFC Draft + +[[!inline pagenames=hurd/what_is_an_os_bootstrap raw=yes feeds=no]] + +Sergey Bugaev proposed: + +The Hurd's current bootstrap, [[Quiet-Boot|hurd/bootstrap]] (a biased +and made-up name), is fragile, hard to debug, and complicated: + +* `Quiet-boot` chokes on misspelled or missing boot arguments. When + this happens, the Hurd bootstrap will likely hang and display + nothing. This is tricky to debug. +* `Quiet-Boot` is hard to change. For instance, when the Hurd + developers added `acpi`, the `pci-arbiter`, and `rumpdisk`, they + struggled to get `Quiet-Boot` working again. +* `Quiet-Boot` forces each bootstrap task to include special bootstrap + logic to work. This limits what is possible during the + bootstrap. For instance, it should be trivial for the Hurd to + support netboot, but `Quiet-Boot` makes it hard to add `nfs`, + `pfinet`, and `isofs` to the bootstrap. +* `Quiet-Boot` hurts other Hurd distributions too. When Guix + developers updated their packaged version of the Hurd, that included + support for SATA drives, a simple misspelled boot argument halted + their progress for a few weeks. + +The alternative `ServerBoot V2` proposal (which was discussed on +[irc](https://logs.guix.gnu.org/hurd/2023-07-18.log) and is similar to +the previously discussed [bootshell +proposal](https://mail-archive.com/bug-hurd@gnu.org/msg26341.html)) +aims to code all or most of the bootstrap specific logic into one +single task (`/hurd/serverboot`). `Serverboot V2` has a number +of enticing advantages: + +* It simplifies the hierarchical dependency of translators during + bootstrap. Developers should be able to re-order and add new + bootstrap translators with minimal work. +* It gives early bootstrap translators like `auth` and `ext2fs` + standard input and output which lets them display boot errors. It + also lets signals work. +* One can trivially use most Hurd translators during the + bootstrap. You just have to link them statically. +* `libmachdev` could be simplified to only expose hardware to + userspace; it might even be possible to remove it entirely. Also + the `pci-arbiter`, `acpi`, and `rumpdisk` could be simplified. +* Developers could remove any bootstrap logic from `libdiskfs`, which + detects the bootstrap filesystem, starts the `exec` server, and + spawns `/hurd/startup`. Instead,`libdiskfs` would only focus on + providing filesystem support. +* If an error happens during early boot, the user could be dropped + into a REPL or mini-console, where he can try to debug the issue. + We might call this `Bootshell V2`, in reference to the original + proposal. This could be written in lisp. Imagine having an + extremely powerful programming language available during bootstrap + that is only [436 bytes!](https://justine.lol/sectorlisp2) +* It would simplify the code for subhurds by removing the logic from + each task that deals with the OS bootstrap. + +Now that you know why we should use `Serverboot V2`, let's get more +detailed. What is `Serverboot V2` ? + +`Serverboot V2` would be an empty filesystem dynamically populated +during bootstrap. It would use a `netfs` like filesystem that will +populate as various bootstrap tasks are started. For example, +`/servers/socket2` will be created once `pfinet` starts. It also +temporarily pretends to be the Hurd process server, `exec`, and `/` +filesystem while providing signals and `stdio`. Let's explain how +`Serverboot V2` will bootstrap the Hurd. + +**FIXME The rest of this needs work.** + +Any bootstrap that the Hurd uses will probably be a little odd, +because there is an awkward and circular startup-dance between +`exec`, `ext2fs`, `startup`, `proc`, `auth`, the `pci-arbiter`, +`rumpdisk`, and `acpi` in which each translator oddly depends on the +other during the bootstrap, as this ascii art shows. + + + pci-arbiter + | + acpi + | + rumpdisk + | + ex2fs -- storeio + / \ + exec startup + / \ + auth proc + + +This means that there is no *perfect* Hurd bootstrap design. Some +designs are better in some ways and worse in others. `Serverboot V2` +would simplify other early bootstrap tasks, but all that complicated +logic would be in one binary. One valid criticism of `Serverboot V2` +is that it will may be a hassle to develop and maintain. In any case, +trying to code the *best* Hurd bootstrap may be a waste of time. In +fact, the Hurd bootstrap has been rewritten several times already. +Our fearless leader, Samuel, feels that rewriting the Hurd bootstrap +every few years may be a waste of time. Now that you understand why +Samuel's discourages a Hurd bootstrap rewrite, let's consider why we +should develop `Serverboot V2`. + +# How ServerBoot V2 will work + +Bootstrap begins when Grub and GNU Mach start some tasks, and then GNU +Mach resumes the not-yet-written +`/hurd/serverboot`. `/hurd/serverboot` is the only task to accept +special ports from the kernel via command line arguments like +`--kernel-task`; `/hurd/serverboot` tries to implement/emulate as much +of the normal Hurd environment for the other bootstrap translators. +In particular, it provides the other translators with `stdio`, which +lets them read/write without having to open the Mach console device. +This means that the various translators will be able to complain about +their bad arguments or other startup errors, which they cannot +currently do. + +`/hurd/serverboot` will provide a basic filesystem with netfs, which +gives the other translators a working `/` directory and `cwd` +ports. For example, `/hurd/serverboot`, would store its port at +`/dev/netdde`. When `/hurd/netdde` starts, it will reply to its +parent with `fsys_startup ()` as normal. + +`/hurd/serverboot` will also emulate the native Hurd process server to +early bootstrap tasks. This will allow early bootstrap tasks to get +the privileged (device master and kernel task) ports via the normal +glibc function `get_privileged_ports (&host_priv, &device_master).` +Other tasks will register their message ports with the emulated +process server. This will allow signals and messaging during the +bootstrap. We can even use the existing mechanisms in glibc to set and +get init ports. For example, when we start the `auth` server, we will +give every task started thus far, their new authentication port via +glibc's `msg_set_init_port ()`. When we start the real proc server, +we query it for proc ports for each of the tasks, and set them the +same way. This lets us migrate from the emulated proc server to the +real one. + +**Fix me: Where does storeio (storeio with** +`device:@/dev/rumpdisk:wd0`**), rumpdisk, and the pci-arbiter come +in?** + +Next, we start `ext2fs`. We reattach all the running translators from +our `netfs` bootstrap filesystem onto the new root. We then send +those translators their new root and cwd ports. This should happen +transparently to the translators themselves! + +# Supporting Netboot + +`Serverboot V2` could trivially support netboot by adding `netdde`, +`pfinet` (or `lwip`), and `isofs` as bootstrap tasks. The bootstrap +task will start the `pci-arbiter`, and `acpi` (FIXME add some more +detail to this sentence). The bootstrap task starts `netdde`, which +will look up any `eth` devices (using the device master port, which it +queries via the fake process server interface), and sends its fsys +control port to the bootstrap task in the regular `fsys_startup +()`. The bootstrap task sets the fsys control port as the translator +on the `/dev/netdde` node in its `netfs` bootstrap fs. Then +`/hurd/serverboot` resumes `pfinet`, which looks up +`/dev/netdde`. Then `pfinet` returns its `fsys` control port to the +bootstrap task, which it sets on `/servers/socket/2`. Then bootstrap +resumes `nfs`, and `nfs` just creates a socket using the regular glibc +socket () call, and that looks up `/servers/socket/2`, and it just +works. **FIXME where does isofs fit in here?** + +Then `nfs` gives its `fsys` control port to `/hurd/serverboot`, which +knows it's the real root filesystem, so it take the netdde's and +pfinet's fsys control ports. Then it calls `file_set_translator ()` +on the nfs on the same paths, so now `/dev/netdde` and +`/servers/socket/2` exist and are accessible both on our bootstrap fs, +and on the new root fs. The bootstrap can then take the root fs to +broadcast a root and cwd port to all other tasks via a +`msg_set_init_port ()`. Now every task is running on the real root fs, +and our little bootstrap fs is no longer used. + +`/hurd/serverboot` can resume the exec server (which is the first +dynamically-linked task) with the real root fs. Then we just +`file_set_translator ()` on the exec server to `/servers/exec`, so +that `nfs` doesn't have to care about this. The bootstrap can now +spawn tasks, instead of resuming ones loaded by Mach and grub, so it +next spawns the `auth` and `proc` servers and gives everyone their +`auth` and `proc` ports. By that point, we have enough of a Unix +environment to call `fork()` and `exec()`. Then the bootstrap tasks +would do the things that `/hurd/startup` used to do, and finally +spawns (or execs) `init / PID 1`. + +With this scheme you will be able to use ext2fs to start to your root +fs via as `/hurd/ext2fs.static /dev/wd0s1`. This eliminates boot +arguments like `--magit-port` and `--next-task`. + +This also simplifies `libmachdev`, which exposes devices to userspace +via some Mach `device_*` RPC calls, which lets the Hurd contain device +drivers instead of GNU Mach. Everything that connects to hardware can +be a `machdev`. + +Additionally, during the `Quiet Boot` bootstrap,`libmachdev` awkwardly +uses `libtrivfs` to create a transient `/` directory, so that the +`pci-arbiter` can mount a netfs on top of it at bootstrap. +`libmachdev` needs `/servers/bus` to mount `/pci,`and it also +needs `/servers` and `/servers/bus` (and `/dev`, and +`/servers/socket`). That complexity could be moved to `ServerbootV2`, +which will create directory nodes at those locations. + +`libmachdev` provides a trivfs that intercepts the `device_open` rpc, +which the `/dev` node uses. It also fakes a root filesystem node, so +you can mount a `netfs` onto it. You still have to implement +`device_read` and `device_write` yourself, but that code runs in +userspace. An example of this can be found in +`rumpdisk/block-rump.c`. + +`libpciaccess` is a special case: it has two modes, the first time it +runs via `pci-arbiter`, it acquires the pci config IO ports and runs +as x86 mode. Every subsequent access of pci becomes a hurdish user of +pci-arbiter. + +`rumpdisk` exposes `/dev/rumpdisk`: + + $ showtrans /dev/rumpdisk + /hurd/rumpdisk + + +# FAQ + +## `Server Boot V2` looks like a ramdisk + a script...? + +Its not quite a ramdisk, its more a netfs translator that +creates a temporary `/`. Its a statically linked binary. I don't +think it differs from a multiboot module. + +## How are the device nodes on the bootstrap netfs attached to each translator? +## How does the first non-bootstrap task get invoked? +## does bootstrap resume it? +## Could we just use a ram disk instead? +## One could stick an unionfs on top of it to load the rest of the system after bootstrap. + +It looks similar to a ramdisk in principle, i.e. it exposes a fs which +lives only in ram, but a ramdisk would not help with early bootstrap. +Namely during early bootstrap, there are no signals or console. +Passing control from from one server to the next via a bootstrap port +is a kludge at best. How many times have you seen the bootstrap +process hang and just sit there? `Serverboot V2` would solve that. +Also, it would allow subhurds to be full hurds without special casing +each task with bootstrap code. It would also clean up `libmachdev`, +and Damien, its author, is in full support. + +## A ramdisk could implement signals and stdio. Isn't that more flexible? + +But if its a ramdisk essentially you have to provide it with a tar +image. Having it live inside a bootstrap task only is +preferable. Also the task could even exit when its done whether you +use an actual ramdisk or not. You still need to write the task that +boots the system. That is different than how it works currently. Also +a ramdisk would have to live in mach, and we want to move things out +of mach. + +Additionally, the bootstrap task will be loaded as the first multiboot +module by grub. It's not a ramdisk, because a ramdisk has to contain +some fs image (with data), and we'd need to parse that format. It +might make sense to steer it more into that direction (and Samuel +seems to have preferred it), because there could potentially be some +config files, or other files that the servers may need to run. I'm not +super fond of that idea. I'd prefer the bootstrap fs to be just a +place where ports (translators) can be placed and looked up. Actually +in my current code it doesn't even use `netfs`, it just implements the +RPCs directly. I'll possibly switch to `netfs` later, or if the +implementation stays simple, I won't use `netfs`. + +## Serverboot V2 just rewrites proc and exec. Why reimplement so much code? + +I don't want to exactly reimplement full `proc` and `exec` servers in the +bootstrap task, it's more of providing very minimal emulation of some +of their functions. I want to implement the two RPCs from the +`proc` interface, one to give a task the privileged ports on request and +one to let the task give me its msg port. That seems fairly simple to +me. + +While we were talking of using netfs, my actual implementation doesn't +even use that, it just implements the RPCs directly (not to suggest I +have anything resembling a complete implementation). Here's some +sample code to give you an idea of what it is like + + + error_t + S_proc_getprivports (struct bootstrap_task *task, + mach_port_t *host_priv, + mach_port_t *device_master) + { + if (!task) + return EOPNOTSUPP; + + if (bootstrap_verbose) + fprintf (stderr, "S_proc_getprivports from %s\n", task->name); + + *host_priv = _hurd_host_priv; + *device_master = _hurd_device_master; + + return 0; + } + + error_t + S_proc_setmsgport (struct bootstrap_task *task, + mach_port_t reply_port, + mach_msg_type_name_t reply_portPoly, + mach_port_t newmsgport, + mach_port_t *oldmsgport, + mach_msg_type_name_t *oldmsgportPoly) + { + if (!task) + return EOPNOTSUPP; + + if (bootstrap_verbose) + fprintf (stderr, "S_proc_setmsgport for %s\n", task->name); + + *oldmsgport = task->msgport; + *oldmsgportPoly = MACH_MSG_TYPE_MOVE_SEND; + + task->msgport = newmsgport; + + return 0; + } + +Yes, it really is just letting tasks fetch the priv ports (so +`get_privileged_ports ()` in glibc works) and set their message ports. +So much for a slippery slope of reimplementing the whole process +server :) + + +## Let's bootstrap like this: initrd, proc, exec, acpi, pci, drivers, +## unionfs+fs with every server executable included in the initrd tarball? + +I don't see how that's better, but you would be able to try something +like that with my plan too. The OS bootstrap needs to start servers +and integrate them into the eventual full hurd system later when the +rest of the system is up. When early servers start, they're running +on bare Mach with no processes, no `auth`, no files or file +descriptors, etc. I plan to make files available immediately (if not +the real fs), and make things progressively more "real" as servers +start up. When we start the root fs, we send everyone their new root +`dir` port. When we start `proc`, we send everyone their new `proc` +port. and so on. At the end, all those tasks we have started in +early boot are full real hurd proceses that are not any different to +the ones you start later, except that they're statically linked, and +not actually `io map`'ed from the root fs, but loaded by Mach/grub +into wired memory. + +# IRC Logs + + <damo22> showtrans /dev/wd0 and you can open() that node and it will + act as a device master port, so you can then `device_open` () devices + (like wd0) inside of it, right? + + oh it's a storeio, that's… cute. that's another translator we'd need + in early boot if we want to boot off /hurd/ext2fs.static /dev/wd0 + + <damo22> We implemented it as a storeio with + device:@/dev/rumpdisk:wd0 + + so the `@` sign makes it use the named file as the device master, right? + + <damo22> the `@` symbol means it looks up the file as the device + master yes. Instead of mach, but the code falls back to looking up + mach, if it cant be found. + + I see it's even implemented in libstore, not in storeio, so it just + does `file_name_lookup ()`, then `device_open` on that. + + <damo22> pci-arbiter also needs acpi because the only way to know the + IRQ of a pci device reliably is to use ACPI parser, so it totally + implements the Mach `device_*` functions. But instead of handling the + RPCs directly, it sets the callbacks into the + `machdev_device_emulations_ops` structure and then libmachdev calls + those. Instead of implementing the RPCs themselves, It abstracts them, + in case you wanted to merge drivers. This would help if you wanted + multiple different devices in the same translator, which is of course + the case inside Mach, the single kernel server does all the devices. + + but that shouldn't be the case for the Hurd translators, right? we'd + just have multiple different translators like your thing with rumpdisk + and rumpusb. + + `<damo22>` i dont know + + ok, so other than those machdev emulation dispatch, libmachdev uses + trivfs and does early bootstrap. pci-arbiter uses it to centralize the + early bootstrap so all the machdevs can use the same code. They chain + together. pci-arbiter creates a netfs on top of the trivfs. How + well does this work if it's not actually used in early bootstrap? + + <damo22> and rumpdisk opens device ("pci"), when each task is resumed, + it inherits a bootstrap port + + and what does it do with that? what kind of device "pci" is? + + <damo22> its the device master for pci, so rumpdisk can call + pci-arbiter rpcs on it + + hm, so I see from the code that it returns the port to the root of its + translator tree actually. Does pci-arbiter have its own rpcs? does it + not just expose an fs tree? + + <damo22> it has rpcs that can be called on each fs node called + "config" per device: hurd/pci.defs. libpciaccess uses these. + + how does that compare to reading and writing the fs node with regular read and write? + + <damo22> so the second and subsequent instances of pciaccess end up + calling into the fs tree of pci-arbiter. you can't call read/write on + pci memory its MMIO, and the io ports need `inb`, `inw`, etc. They + need to be accessed using special accessors, not a bitstream. + + but I can do $ hexdump /servers/bus/pci/0000/00/02/0/config + + <damo22> yes you can on the config file + + how is that different from `pci_conf_read` ? it calls that. + + <damo22> the `pci fs` is implemented to allow these things. + + why is there a need for `pci_conf_read ()` as an RPC then, if you can + instead use `io_read` on the "config" node? + + <damo22> i am not 100% sure. I think it wasn't fully implemented from + the beginning, but you definitely cannot use `io_read ()` on IO + ports. These have explicit x86 instructions to access them + MMIO. maybe, im not sure, but it has absolute physical addressing. + + I don't see how you would do this via `pci.defs` either? + + <damo22> We expose all the device tree of pci as a netfs + filesystem. It is a bus of devices. you may be right. It would be best + to implement pciaccess to just read/write from the filesystem once its + exposed on the netfs. + + yes, the question is: + + 1 is there anything that you can do by using the special RPCs from + pci.defs that you cannot do by using the regular read/write/ls/map + on the exported filsystem tree, + 2 if no, why is there even a need for `pci.defs`, why not always use + the fs? But anyway, that's irrelevant for the question of bootstrap + and libmachdev + + <damo22> There is a need for rpcs for IO ports. + + Could you point me to where rumpdisk does `device_open ("pci")`? grep + doesn't show anything. which rpcs are for the IO ports? + + <damo22> They're not implemented yet we are using raw access I + think. The way it works, libmachdev uses the next port, so it all + chains together: `libmachdev/trivfs_server.c`. + + but where does it call `device_open ("pci")` ? + + <damo22> when the pci task resumes, it has a bootstrap port, which is + passed from previous task. There is no `device_open ("pci")`. or if + its the first task to be resumed, it grabs a bootstrap port from + glibc? im not sure + + ok, so if my plan is implemented how much of `libmachdev` functionality + will still be used / useful? + + <damo22> i dont know. The mach interface? device interface\*. maybe + it will be useless. + + I'd rather you implemented the Mach device RPCs directly, without the + emulation structure, but that's an unrelated change, we can leave that + in for now. + + <damo22> I kind of like the emulation structure as a list of function + pointers, so i can see what needs to be implemented, but that's + neither here nor there. `libmachdev` was a hack to make the bootstrap + work to be honest.…and we'd no longer need that. I would be happy if + it goes away. the new one would be so much better. + + is there anything else I should know about this all? What else could + break if there was no libmachdev and all that? + + <damo22> acpi, pci-arbiter, rumpdisk, rumpusbdisk + + right, let's go through these + + <damo22> The pci-arbiter needs to start first to claim the x86 config + io ports. Then gnumach locks these ports. No one else can use them. + + so it starts and initializes **something** what does it need? the + device master port, clearly, right? that it will get through the + glibc function / the proc API + + <damo22> it needs a /servers/bus and the device master + + <solid_black> + right, so then it just does fsys_startup, and the bootstrap task + places it onto `/servers/bus` (it's not expected to do + `file_set_translator ()` itself, just as when running as a normal + translator) + + <damo22> it exposes a netfs on `/servers/bus/pci` + + <solid_black> so will pci-arbiter still expose mach devices? a mach + device master? or will it only expose an fs tree + pci.defs? + + <damo22> i think just fs tree and pci.defs. should be enough + + <solid_black> ok, so we drop mach dev stuff from pci-arbiter + completely. then acpi starts up, right? what does it need? + + <damo22> It needs access to `pci.defs` and the pci tree. It + accesses that via libpciaccess, which calls a new mode that + accesses the fstree. It looks up `servers/bus/pci`. + + ok, but how does that work now then? + + <damo22> It looks up the right nodes and calls pci.defs on them. + + <solid_black> looks up the right node on what? there's no root + filesystem at that point (in the current scheme) + + `<damo22>` It needs pci access + + that's why I was wondering how it does `device_open ("pci")` + + <damo22> I think libmachdev from pci gives acpi the fsroot. there is a + doc on this. + + so does it set the root node of pci-arbiter as the root dir of acpi? + as in, is acpi effectively chrooted to `/servers/bus/pci`? + + <damo22> i think acpi is chrooted to the parent of /servers. It shares + the same root as pci's trivfs. + + i still don't quite understand how netfs and trivfs within pci-arbiter interact. + + <damo22> you said there would be a fake /. Can't acpi use that? + + <solid_black> yeah, in my plan / the new bootstrap scheme, there'll be + a / from the very start. + + <damo22> ok so acpi can look up /servers/bus/pci, and it will exist. + + and pci-arbiter can really sit on `/servers/bus/pci` (no need for + trivfs there at all) and acpi will just look up + `/servers/bus/pci`. And we do not need to change anything in acpi to + get it to do that. + + And how does it do it now? maybe we'd need to remove some + no-longer-required logic from acpi then? + + <damo22> it looks up device ("pci") if it exists, otherwise it falls + back to `/servers/bus/pci`. + + Ah hold on, maybe I do understand now. currently pci-arbiter exposes + its mach dev master as acpi-s mach dev master. So it looks up + device("pci") and finds it that way. + + <damo22> correct, but it doesnt need that if the `/` exists. + + yeah, we could remove this in the new bootstrap scheme, and just + always open the fs node (or leave it in for compatibility, we'll see + about that). acpi just sits on `/servers/acpi/tables`. + + `rumpdisk` runs next and it needs `/servers/bus/pci`, `pci.defs`, and + `/servers/acpi/tables`, and `acpi.defs`. It exposes `/dev/rumpdisk`. + + Would it make sense to make rumpdisk expose a tree/directory of Hurd + files and not Mach devices? This is not necessary for anything, but + just might be a nice little cleanup. + + <damo22> well, it could expose a tree of block devices, like + `/dev/rumpdisk/ide/1`. + + <solid_black> and then `ln -s /rumpdisk/ide/1 /dev/wd1`. and no need + for an intermediary storeio. plus the Hurd file interface is much + richer than Mach device, you can do fsync for instance. + + <damo22> the rump kernel is bsd under the hood, so needs to be + `/dev/rumpdisk/ide/wd0` + + <solid_black> You can just convert "ide/0" to "/dev/wd0" when + forwarding to the rump part. Not that I object to ide/wd0, but we can + have something more hierarchical in the exposed tree than old-school + unix device naming? Let's not have /dev/sda1. Instead let's have + /dev/sata/0/1, but then we'd still keep the bsd names as symlinks into + the *dev/rumpdisk*… tree + + <damo22> sda sda1 + + <solid_black> good point + + <damo22> 0 0/1 + + <solid_black> well, you can on the Hurd :D and we won't be doing that + either, rumpdisk only exposes the devices, not partitions + + <damo22> well you just implement a block device on the directory? but + that would be confusing for users. + + <solid_black> I'd expect rumpdisk to only expose device nodes, like + /dev/rumpdisk/ide/0, and then we'd have /dev/wd0 being a symlink to + that. And /dev/wd0s1 being a storeio of type part:1:/dev/wd0 or + instead of using that, you could pass that as an option to your fs, + like ext2fs -T typed part:1/dev/wd0 + + <damo22> where is the current hurd bootstrap (QuietBoot) docs hosted? + here: + https://git.savannah.gnu.org/cgit/hurd/web.git/plain/hurd/bootstrap.mdwn + + <solid_black> so yeah, you could do the device tree thing I'm + proposing in rumpdisk, or you could leave it exposing Mach devices and + have a bunch of storeios pointing to that. So anyway, let's say + rumpdisk keeps exposing a single node that acts as a Mach device + master and it sits on /dev/rumpdisk. + + <solid_black> Then we either need a storeio, or we could make ext2fs + use that directly. So we start `/hurd/ext2fs.static -T typed + part:1:@/dev/rumpdisk:wd0`. + + <solid_black> I'll drop all the logic in libdiskfs for detecting if + it's the bootstrap filesystem, and starting the exec server, and + spawning /hurd/startup. It'll just be a library to help create + filesystems. + + <solid_black> After that the bootstrap task migrates all those + translator nodes from the temporary / onto the ext2fs, broadcasts the + root and cwd ports to everyone, and off we go to starting auth and + proc and unix. sounds like it all would work indeed. so we're just + removing libmachdev completely, right? + + <damo22> netdde links to it too. I think it has libmachdevdde + + <solid_black> Also how would you script this thing. Like ideally we'd + want the bootstrap task to follow some sort of script which would say, + for example, + + mkdir /servers + mkdir /servers/bus + settrans /servers/bus/pci ${pci-task} --args-to-pci + mkdir /dev + settrans /dev/netdde ${netdde-task} --args-to-netdde + setroot ${ext2fs-task} --args-to-ext2fs + + <solid_black> and ideally the bootstrap task would implement a REPL + where you'd be able to run these commands interactively (if the + existing script fails for instance). It can be like grub, where it has + a predefined script, and you can do something (press a key combo?) to + instead run your own commands in a repl. or if it fails, it bails out + and drops you into the repl, yes. this gives you **so much more** + visibility into the boot process, because currently it's all scattered + across grub, libdiskfs (resuming exec, spawning /hurd/startup), + /hurd/startup, and various tricky pieces of logic in all of these + servers. + + <solid_black> We could call the mini-repl hurdhelper? If something + fails, you're on your own, at best it prints an error message (if the + failing task manages to open the mach console at that point) Perhaps + we call the new bootstrap proposal Bootstrap. + + <solid_black> When/if this is ready, we'll have to remove libmachdev + and port everything else to work without it. + + <damo22> yes its a great idea. I'm not a fan of lisp either. If i + keep in mind that `/` is available early, then I can just clean up the + other stuff. and assume i have `/`, and the device master can be + accessed with the regular glibc function, and you can printf freely + (no need to open the console). Do i need to run `fsys_startup` ? + + yes, exactly like all translators always do. Well you probably run + netfs_startup or whatever, and it calls that. you're not supposed to + call fsys_getpriv or fsys_init + + <damo22> i think my early attempts at writing translators did not use + these, because i assumed i had `/`. Then i realised i didn\`t. And + libmachdev was born. + + <solid-black> Yes, you should assume you have /, and just do all the + regular things you would do. and if something that you would usually + do doesn't work, we should think of a way to make it work by adding + more stuff in the bootstrap task when it's reasonable to, of + course. and please consider exposing the file tree from rumpdisk, + though that's orthogonal. + + <damo22> you mean a tree of block devices? + + <solid_black> Yes, but each device node would be just a Hurd (device) + file, not a Mach device. i.e. it'd support io_read and io_write, not + device_read and device_write. well I guess you could make it support + both. + + <damo22> isnt that storeio's job? + + <solid_black> if a node only implements the device RPCs, we need a + storeio to turn it into a Hurd file, yes. but if you would implement + the file RPCs directly, there wouldn't be a need for the intermediary + storeio, not that it's important. + + <damo22> but thats writing storeio again. thing is, i dont know at + runtime which devices are exposed by rump. It auto probes them and + prints them out but i cant tell programmatically which ones were + detected, becuause rump knows which devices exist but doesn't expose + it over API in any way. Because it runs as a kernel would with just + one driver set. + + <damo22> Rump is a decent set of drivers. It does not have better + hardware support than Linux drivers (of modern Linux)? Instead Rump is + netbsd in a can, and it's essentially unmaintained upstream + too. However, it still is used it to test kernel modules, but it lacks + makefiles to separate all drivers into modules. BUT using rump is + better than updating / redoing the linux drivers port of DDE, because + netbsd internal kernel API is much much more stable than linux. We + would fall behind in a week with linux. No one would maintain the + linux driver -> hurd port. Also, there is a framework that lets you + compile the netbsd drivers as userspace unikernels: rump. Such a + thing only does not exist for modern Linux. Rump is already good + enough for some things. It could replace netdde. It already works for + ide/sata. + + <damo22> Rump it has its own /dev nodes on a rumpfs, so you can do + something like `rump_ls` it. + + <damo22> Rump is a minimal netbsd kernel. It is just the device + drivers, and a bit of pthreading, and has only the drivers that you + link. So rumpdisk only has the ahci and ide drivers and nothing + else. Additionally rump can detect them off the pci bus. + + <damo22> I will create a branch on + <http://git.zammit.org/hurd-sv.git> with cleaned translators. + + <damo22> solid_black: i almost cleaned up acpi and pci-arbiter but + realised they are missing the shutdown notification when i strip out + libmachdev. + + <solid-black>: "how are the device nodes on the bootstrap netfs attached to + each translator?" – I don't think I understand the question, please + clarify. + + <damo22> I was wondering if the new bootstrap process can resume a fs + task and have all the previous translators wake up and serve their + rpcs. without needing to resume them. we have a problem with the + current design, if you implement what we discussed yesterday, the IO + ports wont work because they are not exposed by pci-arbiter yet. I am + working on it, but its not ready. + + <solid_black> I still don't understand the problem. the bootstrap + task resumes others in order. the root fs task too, eventually, but + not before everything that hash to come up before the root fs task is + ready. + + <damo22> I don't think it needs to be a disk. Literally a trivfs is enough. + + <solid_black> why are I/O ports not exposed by pci-arbiter? why isn't + that in issue with how it works currently then? + + <damo22> solid_black: we are using ioperm() in userspace, but i want + to refactor the io port usage to be granularly accessed. so one day + gnumach can store a bitmap of all io ports and reject more than one + range that overlaps ports that are in use. since only one user of any + port at any time is allowed. i dont know if that will allow users to + share the same io ports, but at least it will prevent users from + clobbering each others hw access. + + <solid_black> damo22: (again, sorry for not understanding the hardware + details), so what would be the issue? when the pci arbiter starts, + doesn't it do all the things it has to do with the I/O ports? + + <damo22> io ports are only accessed in raw method now. Any user can do + ioperm(0, 0xffff, 1) and get access to all of them + + <solid_black> doesn't that require host priv or something like that? + + <damo22> yeh probably. maybe only root can. But i want to allow + unprivileged users to access io ports by requesting exclusive access + to a range. + + <solid_black> I see that ioperm () in glibc uses the device master + port, so yeah, root-only (good) + + `<damo22>` first in locks the port range + + <solid_black> but you're saying that there's someting about these I/O + ports that works today, but would break if we implemented what we + discussed yeasterday? what is it, and why? + + `<damo22>` well it might still work. but there's a lot of changes to + be done in general + + <solid_black> let me try to ask it in a different way then + + <damo22> i just know a few of the specifics because i worked on them. + + <solid_black> As I understand it, you're saying that 1: currently any + root process can request access to any range of I/O ports, and you + also want to allow **unprivileged** processes to get access to ranges + of I/O ports, via a new API of the PCI arbiter (but this is not + implemented yet, right?) + + <damo22> yes + + <solid_black> 2: you're saying that something about this would break / + be different in the new scheme, compared to the current scheme. i + don't understand the 2, and the relation between 1 and 2. + + <damo22> 2 not really, I may have been mistaken it probably will + continue working fine. until i try to implement 1. ioperm calls + `i386_io_perm_create` and `i386_io__perm_modify` in the same system + call. I want to seperate these into the arbiter so the request goes + into pci-arbiter and if it succeeds, then the port is returned to the + caller and the caller can change the port access. + + <solid_black> yes, so what about 2 will break 1 when you try to implement it? + + <damo22> with your new bootstrap, we need `i386_io_perm_*` to be + accessible. im not sure how. is that a mach rpc? + + <solid_black> these are mach rpcs. i386_io_perm_create is an rpc that + you do on device master. + + <damo22> should be ok then + + <solid_black> i386_io_perm_modify you do on you task port. yes, I + don't see how this would be problematic. + + <damo22>: you might find this branch useful + <http://git.zammit.org/hurd-sv.git/log/?h=feat-simplify-bootstrap> + + <solid_black> although: + + 1. I'm not sure whether the task itself should be wiring its memory, + or if the bootstrap task should do it. + 2. why do you request startup notifications if you then never do + anything in `S_startup_dosync`? + + <solid_black> same for essential tasks actaully, that should probably + be done by the bootstrap task and not the translator itself (but we'll + see) + + <solid_black> 1. don't `mach_print`, just `fprintf (stderr, "")` + <solid_black> 2. please always verify the return result of + `mach_port_deallocate` (and similar functions), + typically like this: + + err = mach_port_deallocate (…); + assert_perror_backtrace (err); + + this helps catch nasty bugs. + + <solid_black> 3. I wonder why both acpi and pci have their own + `pcifs_startup` and `acpifs_startup`; can't they use `netfs_startup + ()`? + + `<damo22>` 1. no idea, 2. rumpdisk needed it, but these might + not 3. ACK, 4.ACK, 5. I think they couldnt use the `netfs_startup ()` + before but might be able to now. Anyway, this should get you booting + with your bootstrap translator (without rumpdisk). Rumpdisk seems to + use the `device_* RPC` from `libmachdev` to expose its device. + whereas pci and acpi dont use them for anything except `device_open` + to pass their port to the next translator. I think my latest patch + for io ports will work. but i need to rebuild glibc and libpciaccess + and gnumach. Why does libhurduser need to be in glibc? It's quite + annoying to add an rpc. + + I think i have done gnumach io port locking, and pciaccess, but hurd + part needs work and then to merge it needs a rebuild of glibc because + of hurduser + + <damo22> Why cant libhurduser be part of the hurd package? + + I don't think I understnad enough of this to do a review, but I'd + still like to see the patch if it's available anywhere. + + <damo22> ok i can push to my repos + + <solid_black> glibc needs to use the Hurd RPCs (and implement some, + too), and glibc cannot depend on the Hurd package because the Hurd + package depends on glibc. + + <damo22> lol ok + + <solid_black> As things currently stand, glibc depends on the Hurd + **headers** (including mig defs), but not any Hurd binaries. still, + the cross build process is quite convoluted. I posted about it + somewhere: https://floss.social/@bugaevc/109383703992754691 + + <jpoiret> the manual patching of the build system that's needed to + bootstrap everything is a bit suboptimal. + + <damo22> what if you guys submit patches upstream to glibc to add a + build target to copy the headers or whatever is needed? solid_black: + see + [http://git.zammit.org/{libpciaccess.git,gnumach.git](http://git.zammit.org/%7Blibpciaccess.git,gnumach.git)} + on fix-ioperm branches diff --git a/open_issues/smp.mdwn b/open_issues/smp.mdwn index 6496f388..9820f561 100644 --- a/open_issues/smp.mdwn +++ b/open_issues/smp.mdwn @@ -38,6 +38,57 @@ https://en.wikipedia.org/wiki/File:SMP_-_Symmetric_Multiprocessor_System.svg --> # Current Status +As of September 2024, the SMP support is implemented for i386 with +working internet, because it boots with only one cpu in the default +processor set. The slave processors are temporarily disabled until +they can be safely used per task. We are unable to turn on the full +set at boot time due to race bugs. Debian Hurd provides SMP enabled +GNU Mach kernels. + +# How to test the current SMP support + +The easiest way to test the SMP support, is via the [[qemu +guide|https://www.gnu.org/software/hurd/hurd/running/qemu.html]]. +Once you have the Hurd running you can build an SMP enabled GNU Mach. + + $ git clone git://git.savannah.gnu.org/hurd/gnumach.git + $ cd gnumach + $ autoreconf -i + $ mkdir build + $ cd build + $ ../configure --enable-ncpus=4 --enable-apic --enable-kdb --disable-linux-groups + $ make gnumach.gz + $ su + # mv /boot/gnumach-1.8-486.gz /boot/gnumach-1.8-486.gz.bak + # cp gnumach.gz /boot/gnumach-1.8-486.gz + +You may optionally update `/boot/grub/grub.cfg` change `hd0` to `wd0` +and add `console=com0` + + /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 + +update `/etc/fstab` and update `wd0` instead of `hd0`. + + /dev/wd0s2 / ext2 defaults 0 1 + /dev/wd0s1 none swap sw 0 0 + +You can shutdown via `/sbin/poweroff`. + +start qemu with `-smp 4` and add `-nographic` if you want to use `com0`. + + $ qemu-system-i386 -M q35,accel=kvm -smp 4 -m 2G -net \ + user,hostfwd=tcp::2223-:22 -net nic -hda debian-hurd-VERSION.img \ + -nographic + +To test smp: + + $ sudo /path/to/smp /bin/bash + $ stress -c 7 + +smp.c source can be found +[[here|https://lists.gnu.org/archive/html/bug-hurd/2024-02/msg00088.html]]. + +# What was done to get the 32-bit SMP support The GNU Mach source code includes many special cases for multiprocessor, controlled by #if NCPUS > 1 macro. @@ -95,7 +146,7 @@ have to build an isolated environment to execute the non-thread-safe drivers. ### Task list -1. Implement a routine to detect and identify the processors +1. DONE Implement a routine to detect and identify the processors This routine must check the number of processors, initialize the lapic of BSP (the master processor), and assign a kernel ID for each processor. This kernel @@ -180,7 +231,7 @@ have to build an isolated environment to execute the non-thread-safe drivers. Kernel/APIC array until it finds the same APIC ID. Then it will return the index (Kernel ID) of this position. -2. Implement a routine to initialize the processors +2. DONE Implement a routine to initialize the processors This routine will initialize the lapic of each processor and other structures @@ -191,7 +242,7 @@ have to build an isolated environment to execute the non-thread-safe drivers. Developer Guide, Volume 3. [link](https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf) -3. Implement `intel_startCPU()` +3. DONE Implement `intel_startCPU()` This function will initialize the descriptor tables of the processor specified by the @@ -289,7 +340,7 @@ maillist](https://lists.gnu.org/archive/html/bug-hurd/2018-08/msg00048.html) - [Initial thread in bug-hurd maillist](https://lists.gnu.org/archive/html/bug-hurd/2018-06/msg00048.html) - [SMP in GNU/Hurd FAQ](https://www.gnu.org/software/hurd/faq/smp.html) -- [GNU Mach git repository](http://git.savannah.gnu.org/cgit/hurd/gnumach.git) +- [GNU Mach git repository](https://git.savannah.gnu.org/cgit/hurd/gnumach.git) - [GNU Mach reference manual](https://www.gnu.org/software/hurd/gnumach-doc/) - [**MultiProcessor Specification**](https://pdos.csail.mit.edu/6.828/2011/readings/ia32/MPspec.pdf) - [**ACPI Specification**](http://www.uefi.org/sites/default/files/resources/ACPI%206_2_A_Sept29.pdf) diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn index 99c2cbe9..425a7a82 100644 --- a/open_issues/time.mdwn +++ b/open_issues/time.mdwn @@ -180,7 +180,7 @@ not get a define for `HZ`, which is then defined with a fallback value of 60. <braunr> keep in mind rpctrace doesn't behave like ptrace at all <braunr> it acts as a proxy <braunr> nalaginrut: - http://git.savannah.gnu.org/cgit/hurd/glibc.git/commit/?h=rbraun/getclktck_100_hz&id=90404d6d1aa01f6ce1557841f5a675bb6a30f508 + https://git.savannah.gnu.org/cgit/hurd/glibc.git/commit/?h=rbraun/getclktck_100_hz&id=90404d6d1aa01f6ce1557841f5a675bb6a30f508 <braunr> nalaginrut: you need to add it to the debian eglibc patch list, rebuild the packages, and install the resulting .debs <braunr> if you have trouble doing it, i'll make packages when i have time @@ -786,7 +786,7 @@ not get a define for `HZ`, which is then defined with a fallback value of 60. <braunr> 04:53 < nalaginrut> braunr: BTW, where is the patch/ <braunr> there is hardly anyone here at 5am <braunr> nalaginrut: - http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/clock_t_centiseconds + https://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/clock_t_centiseconds <nalaginrut> braunr: thanks for that, but why not use a constant for 100? <braunr> nalaginrut: i don't know where to define it <braunr> it's glibc, you don't define new stuff mindlessly diff --git a/open_issues/tmpfs.mdwn b/open_issues/tmpfs.mdwn new file mode 100644 index 00000000..9c86db9a --- /dev/null +++ b/open_issues/tmpfs.mdwn @@ -0,0 +1,26 @@ +[[!meta copyright="Copyright © 2025 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_tmpfs]] + +For instance the X server crashes on fvwm startup, or mupdf crashes, +when /run is served by a tmpfs. + +What happens is that they create an shm (shmget, thus a file in +/run/shm/, backed by tmpfs), then map it (shmat, i.e. open/mmap/close of +the file), then remove access to it (shmctl(`IPC_RMID`), i.e. unlink the +file). + +The issue is that even if the file is still mapped, unlinking it from +tmpfs makes tmpfs `mach_port_deallocate` the memobj, which apparently +drops the object completely, and subsequent accesses to the mmap thus +fail with SIGBUS. We'd need a way for tmpfs to know about the +mappings of the pager object, to avoid shutting it down. Or a way to +keep the object alive even if tmpfs has deallocated its port to it. diff --git a/open_issues/todo.mdwn b/open_issues/todo.mdwn index 00eb12d0..0ea5967a 100644 --- a/open_issues/todo.mdwn +++ b/open_issues/todo.mdwn @@ -14,5 +14,5 @@ is included in the section entitled [[!tag open_issue_hurd]] The canonical [TODO -file](http://git.savannah.gnu.org/cgit/hurd/hurd.git/plain/TODO) +file](https://git.savannah.gnu.org/cgit/hurd/hurd.git/plain/TODO) from the git archive. |
