diff options
Diffstat (limited to 'open_issues/serverbootv2.mdwn')
| -rw-r--r-- | open_issues/serverbootv2.mdwn | 901 |
1 files changed, 901 insertions, 0 deletions
diff --git a/open_issues/serverbootv2.mdwn b/open_issues/serverbootv2.mdwn new file mode 100644 index 00000000..60507fab --- /dev/null +++ b/open_issues/serverbootv2.mdwn @@ -0,0 +1,901 @@ +[[!meta copyright="Copyright © 2024 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# ServerBootV2 RFC Draft + +[[!inline pagenames=hurd/what_is_an_os_bootstrap raw=yes feeds=no]] + +Sergey Bugaev proposed: + +The Hurd's current bootstrap, [[Quiet-Boot|hurd/bootstrap]] (a biased +and made-up name), is fragile, hard to debug, and complicated: + +* `Quiet-boot` chokes on misspelled or missing boot arguments. When + this happens, the Hurd bootstrap will likely hang and display + nothing. This is tricky to debug. +* `Quiet-Boot` is hard to change. For instance, when the Hurd + developers added `acpi`, the `pci-arbiter`, and `rumpdisk`, they + struggled to get `Quiet-Boot` working again. +* `Quiet-Boot` forces each bootstrap task to include special bootstrap + logic to work. This limits what is possible during the + bootstrap. For instance, it should be trivial for the Hurd to + support netboot, but `Quiet-Boot` makes it hard to add `nfs`, + `pfinet`, and `isofs` to the bootstrap. +* `Quiet-Boot` hurts other Hurd distributions too. When Guix + developers updated their packaged version of the Hurd, that included + support for SATA drives, a simple misspelled boot argument halted + their progress for a few weeks. + +The alternative `ServerBoot V2` proposal (which was discussed on +[irc](https://logs.guix.gnu.org/hurd/2023-07-18.log) and is similar to +the previously discussed [bootshell +proposal](https://mail-archive.com/bug-hurd@gnu.org/msg26341.html)) +aims to code all or most of the bootstrap specific logic into one +single task (`/hurd/serverboot`). `Serverboot V2` has a number +of enticing advantages: + +* It simplifies the hierarchical dependency of translators during + bootstrap. Developers should be able to re-order and add new + bootstrap translators with minimal work. +* It gives early bootstrap translators like `auth` and `ext2fs` + standard input and output which lets them display boot errors. It + also lets signals work. +* One can trivially use most Hurd translators during the + bootstrap. You just have to link them statically. +* `libmachdev` could be simplified to only expose hardware to + userspace; it might even be possible to remove it entirely. Also + the `pci-arbiter`, `acpi`, and `rumpdisk` could be simplified. +* Developers could remove any bootstrap logic from `libdiskfs`, which + detects the bootstrap filesystem, starts the `exec` server, and + spawns `/hurd/startup`. Instead,`libdiskfs` would only focus on + providing filesystem support. +* If an error happens during early boot, the user could be dropped + into a REPL or mini-console, where he can try to debug the issue. + We might call this `Bootshell V2`, in reference to the original + proposal. This could be written in lisp. Imagine having an + extremely powerful programming language available during bootstrap + that is only [436 bytes!](https://justine.lol/sectorlisp2) +* It would simplify the code for subhurds by removing the logic from + each task that deals with the OS bootstrap. + +Now that you know why we should use `Serverboot V2`, let's get more +detailed. What is `Serverboot V2` ? + +`Serverboot V2` would be an empty filesystem dynamically populated +during bootstrap. It would use a `netfs` like filesystem that will +populate as various bootstrap tasks are started. For example, +`/servers/socket2` will be created once `pfinet` starts. It also +temporarily pretends to be the Hurd process server, `exec`, and `/` +filesystem while providing signals and `stdio`. Let's explain how +`Serverboot V2` will bootstrap the Hurd. + +**FIXME The rest of this needs work.** + +Any bootstrap that the Hurd uses will probably be a little odd, +because there is an awkward and circular startup-dance between +`exec`, `ext2fs`, `startup`, `proc`, `auth`, the `pci-arbiter`, +`rumpdisk`, and `acpi` in which each translator oddly depends on the +other during the bootstrap, as this ascii art shows. + + + pci-arbiter + | + acpi + | + rumpdisk + | + ex2fs -- storeio + / \ + exec startup + / \ + auth proc + + +This means that there is no *perfect* Hurd bootstrap design. Some +designs are better in some ways and worse in others. `Serverboot V2` +would simplify other early bootstrap tasks, but all that complicated +logic would be in one binary. One valid criticism of `Serverboot V2` +is that it will may be a hassle to develop and maintain. In any case, +trying to code the *best* Hurd bootstrap may be a waste of time. In +fact, the Hurd bootstrap has been rewritten several times already. +Our fearless leader, Samuel, feels that rewriting the Hurd bootstrap +every few years may be a waste of time. Now that you understand why +Samuel's discourages a Hurd bootstrap rewrite, let's consider why we +should develop `Serverboot V2`. + +# How ServerBoot V2 will work + +Bootstrap begins when Grub and GNU Mach start some tasks, and then GNU +Mach resumes the not-yet-written +`/hurd/serverboot`. `/hurd/serverboot` is the only task to accept +special ports from the kernel via command line arguments like +`--kernel-task`; `/hurd/serverboot` tries to implement/emulate as much +of the normal Hurd environment for the other bootstrap translators. +In particular, it provides the other translators with `stdio`, which +lets them read/write without having to open the Mach console device. +This means that the various translators will be able to complain about +their bad arguments or other startup errors, which they cannot +currently do. + +`/hurd/serverboot` will provide a basic filesystem with netfs, which +gives the other translators a working `/` directory and `cwd` +ports. For example, `/hurd/serverboot`, would store its port at +`/dev/netdde`. When `/hurd/netdde` starts, it will reply to its +parent with `fsys_startup ()` as normal. + +`/hurd/serverboot` will also emulate the native Hurd process server to +early bootstrap tasks. This will allow early bootstrap tasks to get +the privileged (device master and kernel task) ports via the normal +glibc function `get_privileged_ports (&host_priv, &device_master).` +Other tasks will register their message ports with the emulated +process server. This will allow signals and messaging during the +bootstrap. We can even use the existing mechanisms in glibc to set and +get init ports. For example, when we start the `auth` server, we will +give every task started thus far, their new authentication port via +glibc's `msg_set_init_port ()`. When we start the real proc server, +we query it for proc ports for each of the tasks, and set them the +same way. This lets us migrate from the emulated proc server to the +real one. + +**Fix me: Where does storeio (storeio with** +`device:@/dev/rumpdisk:wd0`**), rumpdisk, and the pci-arbiter come +in?** + +Next, we start `ext2fs`. We reattach all the running translators from +our `netfs` bootstrap filesystem onto the new root. We then send +those translators their new root and cwd ports. This should happen +transparently to the translators themselves! + +# Supporting Netboot + +`Serverboot V2` could trivially support netboot by adding `netdde`, +`pfinet` (or `lwip`), and `isofs` as bootstrap tasks. The bootstrap +task will start the `pci-arbiter`, and `acpi` (FIXME add some more +detail to this sentence). The bootstrap task starts `netdde`, which +will look up any `eth` devices (using the device master port, which it +queries via the fake process server interface), and sends its fsys +control port to the bootstrap task in the regular `fsys_startup +()`. The bootstrap task sets the fsys control port as the translator +on the `/dev/netdde` node in its `netfs` bootstrap fs. Then +`/hurd/serverboot` resumes `pfinet`, which looks up +`/dev/netdde`. Then `pfinet` returns its `fsys` control port to the +bootstrap task, which it sets on `/servers/socket/2`. Then bootstrap +resumes `nfs`, and `nfs` just creates a socket using the regular glibc +socket () call, and that looks up `/servers/socket/2`, and it just +works. **FIXME where does isofs fit in here?** + +Then `nfs` gives its `fsys` control port to `/hurd/serverboot`, which +knows it's the real root filesystem, so it take the netdde's and +pfinet's fsys control ports. Then it calls `file_set_translator ()` +on the nfs on the same paths, so now `/dev/netdde` and +`/servers/socket/2` exist and are accessible both on our bootstrap fs, +and on the new root fs. The bootstrap can then take the root fs to +broadcast a root and cwd port to all other tasks via a +`msg_set_init_port ()`. Now every task is running on the real root fs, +and our little bootstrap fs is no longer used. + +`/hurd/serverboot` can resume the exec server (which is the first +dynamically-linked task) with the real root fs. Then we just +`file_set_translator ()` on the exec server to `/servers/exec`, so +that `nfs` doesn't have to care about this. The bootstrap can now +spawn tasks, instead of resuming ones loaded by Mach and grub, so it +next spawns the `auth` and `proc` servers and gives everyone their +`auth` and `proc` ports. By that point, we have enough of a Unix +environment to call `fork()` and `exec()`. Then the bootstrap tasks +would do the things that `/hurd/startup` used to do, and finally +spawns (or execs) `init / PID 1`. + +With this scheme you will be able to use ext2fs to start to your root +fs via as `/hurd/ext2fs.static /dev/wd0s1`. This eliminates boot +arguments like `--magit-port` and `--next-task`. + +This also simplifies `libmachdev`, which exposes devices to userspace +via some Mach `device_*` RPC calls, which lets the Hurd contain device +drivers instead of GNU Mach. Everything that connects to hardware can +be a `machdev`. + +Additionally, during the `Quiet Boot` bootstrap,`libmachdev` awkwardly +uses `libtrivfs` to create a transient `/` directory, so that the +`pci-arbiter` can mount a netfs on top of it at bootstrap. +`libmachdev` needs `/servers/bus` to mount `/pci,`and it also +needs `/servers` and `/servers/bus` (and `/dev`, and +`/servers/socket`). That complexity could be moved to `ServerbootV2`, +which will create directory nodes at those locations. + +`libmachdev` provides a trivfs that intercepts the `device_open` rpc, +which the `/dev` node uses. It also fakes a root filesystem node, so +you can mount a `netfs` onto it. You still have to implement +`device_read` and `device_write` yourself, but that code runs in +userspace. An example of this can be found in +`rumpdisk/block-rump.c`. + +`libpciaccess` is a special case: it has two modes, the first time it +runs via `pci-arbiter`, it acquires the pci config IO ports and runs +as x86 mode. Every subsequent access of pci becomes a hurdish user of +pci-arbiter. + +`rumpdisk` exposes `/dev/rumpdisk`: + +``` +$ showtrans /dev/rumpdisk + /hurd/rumpdisk +``` + + +# FAQ + +## `Server Boot V2` looks like a ramdisk + a script...? + +Its not quite a ramdisk, its more a netfs translator that +creates a temporary `/`. Its a statically linked binary. I don't +think it differs from a multiboot module. + +## How are the device nodes on the bootstrap netfs attached to each translator? +## How does the first non-bootstrap task get invoked? +## does bootstrap resume it? +## Could we just use a ram disk instead? +## One could stick an unionfs on top of it to load the rest of the system after bootstrap. + +It looks similar to a ramdisk in principle, i.e. it exposes a fs which +lives only in ram, but a ramdisk would not help with early bootstrap. +Namely during early bootstrap, there are no signals or console. +Passing control from from one server to the next via a bootstrap port +is a kludge at best. How many times have you seen the bootstrap +process hang and just sit there? `Serverboot V2` would solve that. +Also, it would allow subhurds to be full hurds without special casing +each task with bootstrap code. It would also clean up `libmachdev`, +and Damien, its author, is in full support. + +## A ramdisk could implement signals and stdio. Isn't that more flexible? + +But if its a ramdisk essentially you have to provide it with a tar +image. Having it live inside a bootstrap task only is +preferable. Also the task could even exit when its done whether you +use an actual ramdisk or not. You still need to write the task that +boots the system. That is different than how it works currently. Also +a ramdisk would have to live in mach, and we want to move things out +of mach. + +Additionally, the bootstrap task will be loaded as the first multiboot +module by grub. It's not a ramdisk, because a ramdisk has to contain +some fs image (with data), and we'd need to parse that format. It +might make sense to steer it more into that direction (and Samuel +seems to have preferred it), because there could potentially be some +config files, or other files that the servers may need to run. I'm not +super fond of that idea. I'd prefer the bootstrap fs to be just a +place where ports (translators) can be placed and looked up. Actually +in my current code it doesn't even use `netfs`, it just implements the +RPCs directly. I'll possibly switch to `netfs` later, or if the +implementation stays simple, I won't use `netfs`. + +## Serverboot V2 just rewrites proc and exec. Why reimplement so much code? + +I don't want to exactly reimplement full `proc` and `exec` servers in the +bootstrap task, it's more of providing very minimal emulation of some +of their functions. I want to implement the two RPCs from the +`proc` interface, one to give a task the privileged ports on request and +one to let the task give me its msg port. That seems fairly simple to +me. + +While we were talking of using netfs, my actual implementation doesn't +even use that, it just implements the RPCs directly (not to suggest I +have anything resembling a complete implementation). Here's some +sample code to give you an idea of what it is like + + + error_t + S_proc_getprivports (struct bootstrap_task *task, + mach_port_t *host_priv, + mach_port_t *device_master) + { + if (!task) + return EOPNOTSUPP; + + if (bootstrap_verbose) + fprintf (stderr, "S_proc_getprivports from %s\n", task->name); + + *host_priv = _hurd_host_priv; + *device_master = _hurd_device_master; + + return 0; + } + + error_t + S_proc_setmsgport (struct bootstrap_task *task, + mach_port_t reply_port, + mach_msg_type_name_t reply_portPoly, + mach_port_t newmsgport, + mach_port_t *oldmsgport, + mach_msg_type_name_t *oldmsgportPoly) + { + if (!task) + return EOPNOTSUPP; + + if (bootstrap_verbose) + fprintf (stderr, "S_proc_setmsgport for %s\n", task->name); + + *oldmsgport = task->msgport; + *oldmsgportPoly = MACH_MSG_TYPE_MOVE_SEND; + + task->msgport = newmsgport; + + return 0; + } + +Yes, it really is just letting tasks fetch the priv ports (so +`get_privileged_ports ()` in glibc works) and set their message ports. +So much for a slippery slope of reimplementing the whole process +server :) + + +## Let's bootstrap like this: initrd, proc, exec, acpi, pci, drivers, +## unionfs+fs with every server executable included in the initrd tarball? + +I don't see how that's better, but you would be able to try something +like that with my plan too. The OS bootstrap needs to start servers +and integrate them into the eventual full hurd system later when the +rest of the system is up. When early servers start, they're running +on bare Mach with no processes, no `auth`, no files or file +descriptors, etc. I plan to make files available immediately (if not +the real fs), and make things progressively more "real" as servers +start up. When we start the root fs, we send everyone their new root +`dir` port. When we start `proc`, we send everyone their new `proc` +port. and so on. At the end, all those tasks we have started in +early boot are full real hurd proceses that are not any different to +the ones you start later, except that they're statically linked, and +not actually `io map`'ed from the root fs, but loaded by Mach/grub +into wired memory. + +# IRC Logs + + <damo22> showtrans /dev/wd0 and you can open() that node and it will + act as a device master port, so you can then `device_open` () devices + (like wd0) inside of it, right? + + oh it's a storeio, that's… cute. that's another translator we'd need + in early boot if we want to boot off /hurd/ext2fs.static /dev/wd0 + + <damo22> We implemented it as a storeio with + device:@/dev/rumpdisk:wd0 + + so the `@` sign makes it use the named file as the device master, right? + + <damo22> the `@` symbol means it looks up the file as the device + master yes. Instead of mach, but the code falls back to looking up + mach, if it cant be found. + + I see it's even implemented in libstore, not in storeio, so it just + does `file_name_lookup ()`, then `device_open` on that. + + <damo22> pci-arbiter also needs acpi because the only way to know the + IRQ of a pci device reliably is to use ACPI parser, so it totally + implements the Mach `device_*` functions. But instead of handling the + RPCs directly, it sets the callbacks into the + `machdev_device_emulations_ops` structure and then libmachdev calls + those. Instead of implementing the RPCs themselves, It abstracts them, + in case you wanted to merge drivers. This would help if you wanted + multiple different devices in the same translator, which is of course + the case inside Mach, the single kernel server does all the devices. + + but that shouldn't be the case for the Hurd translators, right? we'd + just have multiple different translators like your thing with rumpdisk + and rumpusb. + + `<damo22>` i dont know + + ok, so other than those machdev emulation dispatch, libmachdev uses + trivfs and does early bootstrap. pci-arbiter uses it to centralize the + early bootstrap so all the machdevs can use the same code. They chain + together. pci-arbiter creates a netfs on top of the trivfs. How + well does this work if it's not actually used in early bootstrap? + + <damo22> and rumpdisk opens device ("pci"), when each task is resumed, + it inherits a bootstrap port + + and what does it do with that? what kind of device "pci" is? + + <damo22> its the device master for pci, so rumpdisk can call + pci-arbiter rpcs on it + + hm, so I see from the code that it returns the port to the root of its + translator tree actually. Does pci-arbiter have its own rpcs? does it + not just expose an fs tree? + + <damo22> it has rpcs that can be called on each fs node called + "config" per device: hurd/pci.defs. libpciaccess uses these. + + how does that compare to reading and writing the fs node with regular read and write? + + <damo22> so the second and subsequent instances of pciaccess end up + calling into the fs tree of pci-arbiter. you can't call read/write on + pci memory its MMIO, and the io ports need `inb`, `inw`, etc. They + need to be accessed using special accessors, not a bitstream. + + but I can do $ hexdump /servers/bus/pci/0000/00/02/0/config + + <damo22> yes you can on the config file + + how is that different from `pci_conf_read` ? it calls that. + + <damo22> the `pci fs` is implemented to allow these things. + + why is there a need for `pci_conf_read ()` as an RPC then, if you can + instead use `io_read` on the "config" node? + + <damo22> i am not 100% sure. I think it wasn't fully implemented from + the beginning, but you definitely cannot use `io_read ()` on IO + ports. These have explicit x86 instructions to access them + MMIO. maybe, im not sure, but it has absolute physical addressing. + + I don't see how you would do this via `pci.defs` either? + + <damo22> We expose all the device tree of pci as a netfs + filesystem. It is a bus of devices. you may be right. It would be best + to implement pciaccess to just read/write from the filesystem once its + exposed on the netfs. + + yes, the question is: + + 1 is there anything that you can do by using the special RPCs from + pci.defs that you cannot do by using the regular read/write/ls/map + on the exported filsystem tree, + 2 if no, why is there even a need for `pci.defs`, why not always use + the fs? But anyway, that's irrelevant for the question of bootstrap + and libmachdev + + <damo22> There is a need for rpcs for IO ports. + + Could you point me to where rumpdisk does `device_open ("pci")`? grep + doesn't show anything. which rpcs are for the IO ports? + + <damo22> They're not implemented yet we are using raw access I + think. The way it works, libmachdev uses the next port, so it all + chains together: `libmachdev/trivfs_server.c`. + + but where does it call `device_open ("pci")` ? + + <damo22> when the pci task resumes, it has a bootstrap port, which is + passed from previous task. There is no `device_open ("pci")`. or if + its the first task to be resumed, it grabs a bootstrap port from + glibc? im not sure + + ok, so if my plan is implemented how much of `libmachdev` functionality + will still be used / useful? + + <damo22> i dont know. The mach interface? device interface\*. maybe + it will be useless. + + I'd rather you implemented the Mach device RPCs directly, without the + emulation structure, but that's an unrelated change, we can leave that + in for now. + + <damo22> I kind of like the emulation structure as a list of function + pointers, so i can see what needs to be implemented, but that's + neither here nor there. `libmachdev` was a hack to make the bootstrap + work to be honest.…and we'd no longer need that. I would be happy if + it goes away. the new one would be so much better. + + is there anything else I should know about this all? What else could + break if there was no libmachdev and all that? + + <damo22> acpi, pci-arbiter, rumpdisk, rumpusbdisk + + right, let's go through these + + <damo22> The pci-arbiter needs to start first to claim the x86 config + io ports. Then gnumach locks these ports. No one else can use them. + + so it starts and initializes **something** what does it need? the + device master port, clearly, right? that it will get through the + glibc function / the proc API + + <damo22> it needs a /servers/bus and the device master + + <solid_black> + right, so then it just does fsys_startup, and the bootstrap task + places it onto `/servers/bus` (it's not expected to do + `file_set_translator ()` itself, just as when running as a normal + translator) + + <damo22> it exposes a netfs on `/servers/bus/pci` + + <solid_black> so will pci-arbiter still expose mach devices? a mach + device master? or will it only expose an fs tree + pci.defs? + + <damo22> i think just fs tree and pci.defs. should be enough + + <solid_black> ok, so we drop mach dev stuff from pci-arbiter + completely. then acpi starts up, right? what does it need? + + <damo22> It needs access to `pci.defs` and the pci tree. It + accesses that via libpciaccess, which calls a new mode that + accesses the fstree. It looks up `servers/bus/pci`. + + ok, but how does that work now then? + + <damo22> It looks up the right nodes and calls pci.defs on them. + + <solid_black> looks up the right node on what? there's no root + filesystem at that point (in the current scheme) + + `<damo22>` It needs pci access + + that's why I was wondering how it does `device_open ("pci")` + + <damo22> I think libmachdev from pci gives acpi the fsroot. there is a + doc on this. + + so does it set the root node of pci-arbiter as the root dir of acpi? + as in, is acpi effectively chrooted to `/servers/bus/pci`? + + <damo22> i think acpi is chrooted to the parent of /servers. It shares + the same root as pci's trivfs. + + i still don't quite understand how netfs and trivfs within pci-arbiter interact. + + <damo22> you said there would be a fake /. Can't acpi use that? + + <solid_black> yeah, in my plan / the new bootstrap scheme, there'll be + a / from the very start. + + <damo22> ok so acpi can look up /servers/bus/pci, and it will exist. + + and pci-arbiter can really sit on `/servers/bus/pci` (no need for + trivfs there at all) and acpi will just look up + `/servers/bus/pci`. And we do not need to change anything in acpi to + get it to do that. + + And how does it do it now? maybe we'd need to remove some + no-longer-required logic from acpi then? + + <damo22> it looks up device ("pci") if it exists, otherwise it falls + back to `/servers/bus/pci`. + + Ah hold on, maybe I do understand now. currently pci-arbiter exposes + its mach dev master as acpi-s mach dev master. So it looks up + device("pci") and finds it that way. + + <damo22> correct, but it doesnt need that if the `/` exists. + + yeah, we could remove this in the new bootstrap scheme, and just + always open the fs node (or leave it in for compatibility, we'll see + about that). acpi just sits on `/servers/acpi/tables`. + + `rumpdisk` runs next and it needs `/servers/bus/pci`, `pci.defs`, and + `/servers/acpi/tables`, and `acpi.defs`. It exposes `/dev/rumpdisk`. + + Would it make sense to make rumpdisk expose a tree/directory of Hurd + files and not Mach devices? This is not necessary for anything, but + just might be a nice little cleanup. + + <damo22> well, it could expose a tree of block devices, like + `/dev/rumpdisk/ide/1`. + + <solid_black> and then `ln -s /rumpdisk/ide/1 /dev/wd1`. and no need + for an intermediary storeio. plus the Hurd file interface is much + richer than Mach device, you can do fsync for instance. + + <damo22> the rump kernel is bsd under the hood, so needs to be + `/dev/rumpdisk/ide/wd0` + + <solid_black> You can just convert "ide/0" to "/dev/wd0" when + forwarding to the rump part. Not that I object to ide/wd0, but we can + have something more hierarchical in the exposed tree than old-school + unix device naming? Let's not have /dev/sda1. Instead let's have + /dev/sata/0/1, but then we'd still keep the bsd names as symlinks into + the *dev/rumpdisk*… tree + + <damo22> sda sda1 + + <solid_black> good point + + <damo22> 0 0/1 + + <solid_black> well, you can on the Hurd :D and we won't be doing that + either, rumpdisk only exposes the devices, not partitions + + <damo22> well you just implement a block device on the directory? but + that would be confusing for users. + + <solid_black> I'd expect rumpdisk to only expose device nodes, like + /dev/rumpdisk/ide/0, and then we'd have /dev/wd0 being a symlink to + that. And /dev/wd0s1 being a storeio of type part:1:/dev/wd0 or + instead of using that, you could pass that as an option to your fs, + like ext2fs -T typed part:1/dev/wd0 + + <damo22> where is the current hurd bootstrap (QuietBoot) docs hosted? + here: + https://git.savannah.gnu.org/cgit/hurd/web.git/plain/hurd/bootstrap.mdwn + + <solid_black> so yeah, you could do the device tree thing I'm + proposing in rumpdisk, or you could leave it exposing Mach devices and + have a bunch of storeios pointing to that. So anyway, let's say + rumpdisk keeps exposing a single node that acts as a Mach device + master and it sits on /dev/rumpdisk. + + <solid_black> Then we either need a storeio, or we could make ext2fs + use that directly. So we start `/hurd/ext2fs.static -T typed + part:1:@/dev/rumpdisk:wd0`. + + <solid_black> I'll drop all the logic in libdiskfs for detecting if + it's the bootstrap filesystem, and starting the exec server, and + spawning /hurd/startup. It'll just be a library to help create + filesystems. + + <solid_black> After that the bootstrap task migrates all those + translator nodes from the temporary / onto the ext2fs, broadcasts the + root and cwd ports to everyone, and off we go to starting auth and + proc and unix. sounds like it all would work indeed. so we're just + removing libmachdev completely, right? + + <damo22> netdde links to it too. I think it has libmachdevdde + + <solid_black> Also how would you script this thing. Like ideally we'd + want the bootstrap task to follow some sort of script which would say, + for example, + + mkdir /servers + mkdir /servers/bus + settrans /servers/bus/pci ${pci-task} --args-to-pci + mkdir /dev + settrans /dev/netdde ${netdde-task} --args-to-netdde + setroot ${ext2fs-task} --args-to-ext2fs + + <solid_black> and ideally the bootstrap task would implement a REPL + where you'd be able to run these commands interactively (if the + existing script fails for instance). It can be like grub, where it has + a predefined script, and you can do something (press a key combo?) to + instead run your own commands in a repl. or if it fails, it bails out + and drops you into the repl, yes. this gives you **so much more** + visibility into the boot process, because currently it's all scattered + across grub, libdiskfs (resuming exec, spawning /hurd/startup), + /hurd/startup, and various tricky pieces of logic in all of these + servers. + + <solid_black> We could call the mini-repl hurdhelper? If something + fails, you're on your own, at best it prints an error message (if the + failing task manages to open the mach console at that point) Perhaps + we call the new bootstrap proposal Bootstrap. + + <solid_black> When/if this is ready, we'll have to remove libmachdev + and port everything else to work without it. + + <damo22> yes its a great idea. I'm not a fan of lisp either. If i + keep in mind that `/` is available early, then I can just clean up the + other stuff. and assume i have `/`, and the device master can be + accessed with the regular glibc function, and you can printf freely + (no need to open the console). Do i need to run `fsys_startup` ? + + yes, exactly like all translators always do. Well you probably run + netfs_startup or whatever, and it calls that. you're not supposed to + call fsys_getpriv or fsys_init + + <damo22> i think my early attempts at writing translators did not use + these, because i assumed i had `/`. Then i realised i didn\`t. And + libmachdev was born. + + <solid-black> Yes, you should assume you have /, and just do all the + regular things you would do. and if something that you would usually + do doesn't work, we should think of a way to make it work by adding + more stuff in the bootstrap task when it's reasonable to, of + course. and please consider exposing the file tree from rumpdisk, + though that's orthogonal. + + <damo22> you mean a tree of block devices? + + <solid_black> Yes, but each device node would be just a Hurd (device) + file, not a Mach device. i.e. it'd support io_read and io_write, not + device_read and device_write. well I guess you could make it support + both. + + <damo22> isnt that storeio's job? + + <solid_black> if a node only implements the device RPCs, we need a + storeio to turn it into a Hurd file, yes. but if you would implement + the file RPCs directly, there wouldn't be a need for the intermediary + storeio, not that it's important. + + <damo22> but thats writing storeio again. thing is, i dont know at + runtime which devices are exposed by rump. It auto probes them and + prints them out but i cant tell programmatically which ones were + detected, becuause rump knows which devices exist but doesn't expose + it over API in any way. Because it runs as a kernel would with just + one driver set. + + <damo22> Rump is a decent set of drivers. It does not have better + hardware support than Linux drivers (of modern Linux)? Instead Rump is + netbsd in a can, and it's essentially unmaintained upstream + too. However, it still is used it to test kernel modules, but it lacks + makefiles to separate all drivers into modules. BUT using rump is + better than updating / redoing the linux drivers port of DDE, because + netbsd internal kernel API is much much more stable than linux. We + would fall behind in a week with linux. No one would maintain the + linux driver -> hurd port. Also, there is a framework that lets you + compile the netbsd drivers as userspace unikernels: rump. Such a + thing only does not exist for modern Linux. Rump is already good + enough for some things. It could replace netdde. It already works for + ide/sata. + + <damo22> Rump it has its own /dev nodes on a rumpfs, so you can do + something like `rump_ls` it. + + <damo22> Rump is a minimal netbsd kernel. It is just the device + drivers, and a bit of pthreading, and has only the drivers that you + link. So rumpdisk only has the ahci and ide drivers and nothing + else. Additionally rump can detect them off the pci bus. + + <damo22> I will create a branch on + <http://git.zammit.org/hurd-sv.git> with cleaned translators. + + <damo22> solid_black: i almost cleaned up acpi and pci-arbiter but + realised they are missing the shutdown notification when i strip out + libmachdev. + + <solid-black>: "how are the device nodes on the bootstrap netfs attached to + each translator?" – I don't think I understand the question, please + clarify. + + <damo22> I was wondering if the new bootstrap process can resume a fs + task and have all the previous translators wake up and serve their + rpcs. without needing to resume them. we have a problem with the + current design, if you implement what we discussed yesterday, the IO + ports wont work because they are not exposed by pci-arbiter yet. I am + working on it, but its not ready. + + <solid_black> I still don't understand the problem. the bootstrap + task resumes others in order. the root fs task too, eventually, but + not before everything that hash to come up before the root fs task is + ready. + + <damo22> I don't think it needs to be a disk. Literally a trivfs is enough. + + <solid_black> why are I/O ports not exposed by pci-arbiter? why isn't + that in issue with how it works currently then? + + <damo22> solid_black: we are using ioperm() in userspace, but i want + to refactor the io port usage to be granularly accessed. so one day + gnumach can store a bitmap of all io ports and reject more than one + range that overlaps ports that are in use. since only one user of any + port at any time is allowed. i dont know if that will allow users to + share the same io ports, but at least it will prevent users from + clobbering each others hw access. + + <solid_black> damo22: (again, sorry for not understanding the hardware + details), so what would be the issue? when the pci arbiter starts, + doesn't it do all the things it has to do with the I/O ports? + + <damo22> io ports are only accessed in raw method now. Any user can do + ioperm(0, 0xffff, 1) and get access to all of them + + <solid_black> doesn't that require host priv or something like that? + + <damo22> yeh probably. maybe only root can. But i want to allow + unprivileged users to access io ports by requesting exclusive access + to a range. + + <solid_black> I see that ioperm () in glibc uses the device master + port, so yeah, root-only (good) + + `<damo22>` first in locks the port range + + <solid_black> but you're saying that there's someting about these I/O + ports that works today, but would break if we implemented what we + discussed yeasterday? what is it, and why? + + `<damo22>` well it might still work. but there's a lot of changes to + be done in general + + <solid_black> let me try to ask it in a different way then + + <damo22> i just know a few of the specifics because i worked on them. + + <solid_black> As I understand it, you're saying that 1: currently any + root process can request access to any range of I/O ports, and you + also want to allow **unprivileged** processes to get access to ranges + of I/O ports, via a new API of the PCI arbiter (but this is not + implemented yet, right?) + + <damo22> yes + + <solid_black> 2: you're saying that something about this would break / + be different in the new scheme, compared to the current scheme. i + don't understand the 2, and the relation between 1 and 2. + + <damo22> 2 not really, I may have been mistaken it probably will + continue working fine. until i try to implement 1. ioperm calls + `i386_io_perm_create` and `i386_io__perm_modify` in the same system + call. I want to seperate these into the arbiter so the request goes + into pci-arbiter and if it succeeds, then the port is returned to the + caller and the caller can change the port access. + + <solid_black> yes, so what about 2 will break 1 when you try to implement it? + + <damo22> with your new bootstrap, we need `i386_io_perm_*` to be + accessible. im not sure how. is that a mach rpc? + + <solid_black> these are mach rpcs. i386_io_perm_create is an rpc that + you do on device master. + + <damo22> should be ok then + + <solid_black> i386_io_perm_modify you do on you task port. yes, I + don't see how this would be problematic. + + <damo22>: you might find this branch useful + <http://git.zammit.org/hurd-sv.git/log/?h=feat-simplify-bootstrap> + + <solid_black> although: + + 1. I'm not sure whether the task itself should be wiring its memory, + or if the bootstrap task should do it. + 2. why do you request startup notifications if you then never do + anything in `S_startup_dosync`? + + <solid_black> same for essential tasks actaully, that should probably + be done by the bootstrap task and not the translator itself (but we'll + see) + + <solid_black> 1. don't `mach_print`, just `fprintf (stderr, "")` + <solid_black> 2. please always verify the return result of + `mach_port_deallocate` (and similar functions), + typically like this: + + err = mach_port_deallocate (…); + assert_perror_backtrace (err); + + this helps catch nasty bugs. + + <solid_black> 3. I wonder why both acpi and pci have their own + `pcifs_startup` and `acpifs_startup`; can't they use `netfs_startup + ()`? + + `<damo22>` 1. no idea, 2. rumpdisk needed it, but these might + not 3. ACK, 4.ACK, 5. I think they couldnt use the `netfs_startup ()` + before but might be able to now. Anyway, this should get you booting + with your bootstrap translator (without rumpdisk). Rumpdisk seems to + use the `device_* RPC` from `libmachdev` to expose its device. + whereas pci and acpi dont use them for anything except `device_open` + to pass their port to the next translator. I think my latest patch + for io ports will work. but i need to rebuild glibc and libpciaccess + and gnumach. Why does libhurduser need to be in glibc? It's quite + annoying to add an rpc. + + I think i have done gnumach io port locking, and pciaccess, but hurd + part needs work and then to merge it needs a rebuild of glibc because + of hurduser + + <damo22> Why cant libhurduser be part of the hurd package? + + I don't think I understnad enough of this to do a review, but I'd + still like to see the patch if it's available anywhere. + + <damo22> ok i can push to my repos + + <solid_black> glibc needs to use the Hurd RPCs (and implement some, + too), and glibc cannot depend on the Hurd package because the Hurd + package depends on glibc. + + <damo22> lol ok + + <solid_black> As things currently stand, glibc depends on the Hurd + **headers** (including mig defs), but not any Hurd binaries. still, + the cross build process is quite convoluted. I posted about it + somewhere: https://floss.social/@bugaevc/109383703992754691 + + <jpoiret> the manual patching of the build system that's needed to + bootstrap everything is a bit suboptimal. + + <damo22> what if you guys submit patches upstream to glibc to add a + build target to copy the headers or whatever is needed? solid_black: + see + [http://git.zammit.org/{libpciaccess.git,gnumach.git](http://git.zammit.org/%7Blibpciaccess.git,gnumach.git)} + on fix-ioperm branches |